TLDR: SymLight is a novel framework that uses Monte Carlo Tree Search to discover interpretable and deployable symbolic policies for adaptive traffic signal control. It addresses the limitations of Deep Reinforcement Learning (DRL) in terms of transparency and hardware deployment. SymLight generates human-understandable policies that outperform various baselines on real-world datasets, while being lightweight enough for resource-constrained edge devices and aligning training objectives with real-world performance metrics.
Adaptive Traffic Signal Control (ATSC) is a crucial component of modern urban infrastructure, aiming to optimize traffic flow and reduce congestion. While Deep Reinforcement Learning (DRL) has shown great promise in this area, its real-world adoption has been surprisingly low. This is primarily due to two significant challenges: the lack of interpretability in DRL’s ‘black-box’ neural network policies and the difficulty of deploying these complex models on resource-limited edge devices commonly found at intersections.
A new research paper introduces SymLight, an innovative framework designed to overcome these limitations. SymLight focuses on discovering inherently interpretable and deployable symbolic policies for traffic signal control, offering a fresh perspective on making smart traffic management both effective and transparent.
The Problem with Current Adaptive Traffic Systems
Traditional traffic control methods, like fixed-time or manually designed heuristic strategies, are easy to understand but struggle with the dynamic and complex nature of modern traffic. DRL-based systems, while powerful, often generate policies that are opaque. This ‘black-box’ nature makes it difficult for traffic engineers to validate decisions, troubleshoot issues, or for authorities to build public trust. Furthermore, deploying these computationally intensive neural policies on low-cost edge devices leads to significant latency and memory constraints, often requiring performance-degrading model compression.
Another critical issue is the misalignment between training objectives and real-world evaluation metrics. DRL models are often optimized for local metrics, while public agencies evaluate systems based on broader objectives like total throughput or average travel time. This mismatch can lead to policies that perform well in simulations but fall short in practical deployment.
Introducing SymLight: A Clearer Path to Traffic Control
SymLight addresses these challenges by formulating traffic signal control as a learnable symbolic priority function. Instead of complex neural networks, SymLight uses explicit symbolic expressions as its policy representation. This priority function takes current traffic features as input and calculates a priority score for each traffic signal phase, ultimately directing which phase turns green next.
The framework employs Monte Carlo Tree Search (MCTS) to explore the vast space of possible symbolic expressions, aiming to identify high-performing priority functions. A key advantage of SymLight is its ability to directly use global, system-level objectives as the reward signal during training, ensuring that what is optimized during learning aligns with real-world performance goals.
How SymLight Works
SymLight’s approach is built on several clever design choices:
- Interpretable Policies: The core of SymLight is its symbolic priority function. These functions are human-readable mathematical expressions, making the decision-making process transparent and verifiable by experts. For example, a policy might be as simple as “incoming lane occupancy multiplied by vehicles in the incoming lane.”
- Efficient Search: To manage the complexity of searching for optimal symbolic expressions, SymLight uses a concise yet expressive representation for its priority functions. It also introduces a Probabilistic Structural Rollout (PSR) strategy within MCTS, which learns from previously discovered high-quality policies to guide the search more effectively.
- Comprehensive Traffic Features: SymLight’s priority function considers eight key traffic features from incoming and outgoing lanes, such as the number of waiting vehicles, total vehicle count, and lane occupancy ratio. These features are easily obtainable from standard road sensors.
- Adaptive Reward Shaping: To ensure stable learning across different traffic environments, SymLight normalizes rewards based on the best performance observed during the search, making the algorithm robust without requiring extensive hyperparameter tuning.
Real-World Performance and Benefits
Extensive experiments on six real-world traffic datasets from cities like Hangzhou, Los Angeles, and Manhattan demonstrate SymLight’s superior performance. It consistently outperforms traditional, DRL-based, and other practical methods in terms of average travel time and throughput. For instance, in the Los Angeles scenario, SymLight achieved an average travel time of 410.28 seconds per vehicle, significantly better than many baselines.
Beyond performance, SymLight delivers on its promise of interpretability and deployability. The generated policies are not only effective but also human-understandable, allowing traffic engineers to gain valuable insights into why certain decisions are made. This transparency also helps in identifying the most crucial traffic features for different scenarios, such as the consistent importance of incoming lane features.
Crucially, SymLight’s symbolic policies are lightweight, requiring significantly less computation time and memory compared to neural network-based approaches. This makes them highly suitable for deployment on existing, resource-constrained edge devices like those found in traffic signal controllers, bridging the gap between advanced research and practical application.
The framework also demonstrates strong generalization capabilities, meaning policies trained in one environment can be effectively applied to unseen scenarios without retraining, further enhancing its real-world utility.
Also Read:
- AgentSUMO: Simplifying Traffic Simulation for Urban Planners
- Optimizing City Traffic: A Balanced Approach to Efficiency, Fairness, and Privacy
Conclusion
SymLight represents a significant step forward in adaptive traffic signal control. By combining the power of Monte Carlo Tree Search with interpretable symbolic policy representations, it offers a solution that is not only highly effective but also transparent, verifiable, and readily deployable on existing infrastructure. This approach promises to make our urban traffic systems smarter, more efficient, and easier to manage. You can read more about this innovative research in the full paper available here.
