TLDR: HLSMAC is a new StarCraft II benchmark for multi-agent reinforcement learning that focuses on high-level strategic decision-making, moving beyond micromanagement. It features 12 scenarios based on the ancient Chinese “Thirty-Six Stratagems,” incorporating larger maps, expanded unit abilities, and diverse opponent behaviors. The benchmark also introduces novel metrics beyond win rate to assess strategic intelligence, revealing that existing AI methods struggle with these complex challenges and that traditional metrics are insufficient for comprehensive evaluation.
Multi-agent reinforcement learning (MARL) has seen significant advancements, largely thanks to various benchmarks that test AI in cooperative, competitive, and mixed environments. StarCraft II, a real-time strategy game, has been a particularly popular platform for these challenges, with environments like the StarCraft Multi-Agent Challenge (SMAC) driving much of the progress. However, existing benchmarks, including SMAC, primarily focus on “micromanagement”—the fine-grained control of individual units. This narrow focus limits the ability to truly evaluate an AI’s capacity for high-level strategic thinking.
To address this crucial gap, researchers have introduced HLSMAC (StarCraft Multi-Agent Challenge for High-Level Strategic Decision-Making). This innovative benchmark aims to push the boundaries of MARL by focusing on complex, human-like strategic reasoning. HLSMAC is built upon 12 carefully designed StarCraft II scenarios, each inspired by classical stratagems from the ancient Chinese “Thirty-Six Stratagems.” These scenarios are crafted to challenge AI agents with diverse strategic elements, such as tactical maneuvering, precise timing, and even deception, moving beyond simple unit control to assess true strategic intelligence.
The design of HLSMAC scenarios incorporates several key features to foster high-level strategic decision-making. Firstly, the maps are significantly larger than those in previous benchmarks, offering more space for movement, diverse routes, and additional combat zones. This encourages agents to utilize spatial and terrain features for strategic advantage rather than just focusing on close-quarters combat. Secondly, units and structures in HLSMAC have expanded abilities. For instance, Zerglings can “Burrow” for ambushes, and Sentries can cast “Hallucination” to create fake allies, introducing elements of deception and long-term planning into the action space. Thirdly, diverse opponent policies are implemented using StarCraft II’s built-in trigger system, allowing for realistic enemy responses like aggressive attacks or strategic retreats based on game conditions. Finally, game termination conditions are redefined beyond simply eliminating all enemy units. Success might involve destroying critical structures or maintaining unit survival for specific periods, encouraging agents to formulate overarching plans rather than just engaging in exhaustive confrontation.
The integration of human strategic wisdom is a cornerstone of HLSMAC. Each of the 12 scenarios is named after a specific stratagem and designed to embody its principles. For example, the “Besiege Wei to Rescue Zhao” scenario (wwjz) challenges agents to attack the enemy’s home base to force them to abandon their assault on the player’s base, mirroring the ancient tactic of striking at what the enemy holds dear. Another scenario, “Lure Your Enemy onto the Roof, Then Take Away the Ladder” (swct), requires agents to transport Sentries to high ground and use “Force Fields” to trap and destroy retreating enemies. The “Kill with a Borrowed Sword” scenario (jdsr) demands the Infestor unit to use “Neural Parasite” to control a powerful enemy unit, turning its strength against its own allies. These examples highlight how HLSMAC translates abstract strategic concepts into concrete in-game challenges.
Beyond traditional win rates, HLSMAC introduces novel metrics to comprehensively evaluate strategic intelligence. These include Critical Target Advancement (measuring movement towards key enemy targets), Ability Utilization Frequency (tracking the use of special unit abilities), Critical Target Damage (assessing damage to crucial enemy structures), and Unit Survival Rate (evaluating the ability to preserve units). These metrics provide a richer, multi-dimensional assessment of an agent’s performance, offering insights into how well an AI understands and executes strategic plans, rather than just whether it wins or loses.
Also Read:
- Enhancing Multi-Agent Coordination with K-Level Policy Gradients
- Simplifying Multi-Agent Cooperation Through Self-Supervised Goal-Reaching
Initial experiments with 21 state-of-the-art MARL algorithms and LLM-based agents (like GPT-3.5) demonstrate that HLSMAC presents significant challenges. Nearly 80% of algorithm-scenario combinations resulted in zero win rates, and LLM-based agents struggled across all scenarios, despite showing some limited tactical understanding. The research also found that traditional win-rate metrics alone are insufficient to capture human-like strategic decision-making, as some algorithms achieved high win rates without truly following the intended stratagems. The new metrics, however, proved valuable in providing deeper insights into strategic performance and distinguishing purposeful ability usage from aimless actions. This robust testbed is expected to drive significant advancements in multi-agent strategic decision-making. For more in-depth information, you can refer to the full research paper: HLSMAC: A New StarCraft Multi-Agent Challenge for High-Level Strategic Decision-Making.
