TLDR: This research introduces a novel method using unsupervised CNN-LSTM autoencoders to identify distinct play styles in AI agents within the MicroRTS game environment. By directly analyzing low-level game data, the approach eliminates the need for manual feature engineering, leading to more generalized and unbiased play style identification and offering insights into AI agent behaviors.
Understanding how players engage with games, often referred to as ‘play style identification,’ is a crucial area for game designers and artificial intelligence (AI) developers. It can lead to more personalized game experiences, dynamically adjusting difficulty or content to keep players engaged. Furthermore, identifying distinct play styles can significantly improve game-playing AI agents, enabling them to exhibit more diverse and human-like behaviors through techniques like imitation learning.
Historically, identifying play styles has often relied on human expertise to create ‘handcrafted features’ from game data. This involves defining specific metrics or patterns that are believed to represent different styles. While effective to some extent, this approach can introduce biases based on the designer’s assumptions and may not generalize well across different games or even different levels within the same game. More recent methods have tried to incorporate the sequential nature of game play, but still often require some level of abstraction from the raw data.
A New Approach to Play Style Identification
A recent research paper, titled “Play Style Identification Using Low-Level Representations of Play Traces in MicroRTS,” introduces a novel method that aims to overcome these limitations. The authors, Ruizhe Yu Xia, Jeremy Gow, and Simon Lucas from Queen Mary University of London, propose using unsupervised deep learning models, specifically CNN-LSTM autoencoders, to automatically learn meaningful representations directly from raw, low-level game data. This approach significantly reduces the reliance on human domain knowledge and its associated biases, allowing play styles to emerge naturally from the data itself.
The research focuses on MicroRTS, a lightweight real-time strategy (RTS) game environment popular for AI research. MicroRTS presents typical RTS challenges like managing units simultaneously and making real-time decisions in a complex action space. A yearly competition for AI agents in MicroRTS has led to the development of many strong AI players, each often exhibiting a distinct play style, making it an ideal environment to test play style identification methods.
How the Method Works
The core of the method involves collecting extensive game data, or ‘play traces,’ from various AI agents playing MicroRTS. For their experiments, the researchers recorded games played by 13 different AI agents, including well-known ones like WorkerRush, NaiveMCTS, and MixedBot. The data captures both the game ‘states’ (what’s happening on the map) and the ‘actions’ taken by the players, represented in a detailed, low-level tensor format.
This raw data is then fed into an autoencoder model. An autoencoder is a type of neural network designed to learn a compressed, ‘latent’ representation of its input data. It consists of two main parts: an encoder, which compresses the input into a smaller representation, and a decoder, which tries to reconstruct the original input from this compressed representation. The CNN-LSTM architecture is particularly suited for sequential data like game traces, with Convolutional Neural Networks (CNNs) processing individual game frames and Long Short-Term Memory (LSTM) networks handling the temporal sequence of these frames.
The researchers trained three types of autoencoders: one that learned from actions only, one from states only, and a ‘joint’ model that combined both. Once trained, the encoder part of the model was used to transform the play traces into their latent representations. These high-dimensional latent vectors were then reduced in dimensionality using Principal Component Analysis (PCA) and clustered using k-means, a common unsupervised clustering algorithm, to group similar play styles together.
Key Findings and Insights
The results showed that the models trained on actions-only and the joint (actions and states) data performed best in identifying distinct play styles, outperforming the states-only model and a baseline using traditional handcrafted features. This suggests that player actions are particularly informative for play style identification in this context.
A qualitative analysis of the clustered play traces revealed fascinating insights into the AI agents’ behaviors:
- The clustering algorithm successfully merged play traces from ‘MixedBot’ and ‘Tiamat,’ which was expected because MixedBot is an ensemble AI that includes Tiamat as one of its components.
- Some agents that were quickly defeated by aggressive strategies (like ‘WorkerRush’) were clustered separately. This indicated that they didn’t have enough time to fully exhibit their intended play style before the game ended.
- Even strong agents like ‘mayari’ showed some unexpected behaviors. While most of its traces clustered together, showcasing optimized resource gathering and unit production, some were grouped with ‘random’ variants. Upon inspection, this was traced back to specific pathing issues early in the game that caused erratic movements, even though the agent would correct itself later.
- The ‘Droplet’ agent, known to be a ‘WorkerRush’ strategy enhanced with Monte Carlo Tree Search, had its traces split between the ‘WorkerRush’ cluster and the ‘NaiveMCTS’ cluster, aligning with its hybrid design.
- Traces from agents that failed or crashed during data collection were appropriately clustered with ‘PassiveAI,’ as they exhibited minimal or no meaningful actions.
This qualitative exploration highlights the power of this unsupervised approach: it not only identifies clusters of play styles but also helps pinpoint specific behavioral quirks or failings within the AI agents themselves, offering valuable debugging and improvement opportunities.
Also Read:
- Advancing Multi-Agent Intelligence with Generative AI
- Enhancing Anomaly Detection in Sensor Networks with Causal Reinforcement Learning
Conclusion and Future Directions
This research demonstrates a powerful method for identifying play styles directly from low-level game data, significantly reducing the need for manual feature engineering and its inherent biases. By allowing patterns to emerge naturally, the approach offers better generalizability across different game environments and ensures that identified styles are truly reflective of player behavior.
The latent space learned by the autoencoder also serves as a valuable tool for analyzing and understanding play traces. The authors suggest that future work could extend this method to commercial games, explore more advanced autoencoder structures like Variational Autoencoders (VAEs), and utilize transformers to handle the larger datasets and higher dimensions typical of modern game analytics.
You can read the full research paper here.
