TLDR: FLAS (Forecasted Load Auto-Scaling) is a novel auto-scaler for distributed cloud services that merges proactive and reactive scaling strategies. It uses predictive models to anticipate SLA parameter trends and a reactive system to estimate performance from resource usage, reducing instrumentation. Evaluated with a content-based publish-subscribe system (E-SilboPS), FLAS achieved over 99% SLA compliance, outperforming individual scaling methods and reducing resource over-provisioning compared to traditional reactive approaches.
Cloud computing has become the backbone for most modern technologies, largely due to its ability to scale resources up or down as needed, a characteristic known as elasticity. This dynamic adjustment is managed by systems called auto-scalers, which acquire and release resources to maintain agreed-upon service levels (SLAs). However, designing effective auto-scalers is complex, as they need to prevent both over-provisioning (wasting resources and increasing cost) and under-provisioning (leading to service degradation and SLA violations).
A new approach, called FLAS (Forecasted Load Auto-Scaling), has been introduced to tackle these challenges. FLAS is an auto-scaler for distributed services that intelligently combines the strengths of both proactive and reactive scaling methods. This means it tries to anticipate future needs while also having a robust plan for unexpected changes.
Key Innovations of FLAS
FLAS brings two main novelties to the table. First, it uses a predictive model to forecast trends in high-level metrics, such as response time or throughput. This allows the system to foresee potential issues with service level agreements and take action before problems arise. Second, it includes a reactive contingency system that estimates high-level metrics from low-level resource usage data. This reduces the need for extensive system instrumentation, making FLAS less intrusive and adaptable to various applications without specific modifications.
The researchers implemented FLAS for a content-based publish-subscribe middleware called E-SilboPS, which is crucial for event-driven architectures. This is notable as it’s the first auto-scaling system designed for such complex distributed systems, though FLAS is generic enough to be applied to other distributed services. The system was rigorously evaluated using various test cases, including worst-case scenarios, demonstrating its effectiveness in maintaining performance requirements over 99% of the time.
How FLAS Works: An Architectural Overview
FLAS operates in two main phases: a monitoring phase and an auto-scaling phase. During monitoring, it gathers data on workload, its trends over time, low-level resource behavior, and high-performance variables relevant to SLAs. This data is then used to generate predictive models. In the auto-scaling phase, these models provide predictions to a ‘Decider’ module, which makes the ultimate decisions on scaling actions.
The architecture of FLAS comprises four functional modules:
- Scaling Time Forecaster: Predicts how long a scaling action will take based on the system’s workload.
- Workload Trend Forecaster: Forecasts the trend of system performance (e.g., response time) over a future time horizon, helping to anticipate saturation points.
- Performance Forecaster: Establishes relationships between low-level resource usage (like CPU or memory) and high-level performance metrics (like response time). This is crucial for identifying bottlenecks and determining which resources to scale.
- Decider: This is the brain of FLAS. It synthesizes information from the other modules to determine if a scaling action is necessary. It first checks proactive predictions of performance trends and then, as a contingency, evaluates current estimated performance against predefined thresholds, acting reactively if needed.
Evaluation and Results
The evaluation of FLAS involved synthetic workloads designed to simulate both typical and extreme scenarios. The goal was to minimize the time spent violating the performance SLA, which was set at a maximum response time of 1 second. The results showed that FLAS consistently achieved high SLA compliance, with violations occurring less than 1% of the time, even in challenging situations.
A significant finding was that FLAS outperformed auto-scaling techniques that relied solely on either proactive or reactive methods. It also achieved similar SLA compliance to traditional threshold-based reactive auto-scalers but with significantly less over-provisioning of resources. This translates directly into lower economic costs and reduced energy consumption, making FLAS a more efficient solution.
The modular design of FLAS allows it to be adapted to different distributed systems by adjusting a few configuration parameters and even replacing its predictive models. This flexibility ensures its broad applicability across various cloud environments. For more technical details, you can refer to the original research paper here.
Also Read:
- AI-Powered Design for Cloud Systems: LLMs and Simulators Optimize Distributed Architectures
- Optimizing AI Inference: A 3D Approach to Balancing Performance, Cost, and Speed
Future Directions
While FLAS currently scales resources by doubling or halving them, future work aims to develop a module that predicts the minimum required configuration more precisely. Researchers are also working on improving workload trend prediction for non-stationary workloads and refining the scaling time prediction by incorporating future load forecasts. These enhancements will further solidify FLAS’s position as a leading auto-scaling solution for distributed services.
