TLDR: Researchers successfully trained AI agents to play Dark Souls, a challenging action RPG, using Neuroevolution of Augmenting Topologies (NEAT) and raw pixel input. They developed a Python framework (DSAPI) to extract game data via computer vision. The agents achieved up to a 35% success rate against the Asylum Demon, demonstrating NEAT’s potential for complex, visually-driven game environments without explicit game-state information.
A recent research paper explores a groundbreaking approach to teaching artificial intelligence to play the notoriously difficult video game, Dark Souls. Unlike traditional methods that rely on explicit game data, this study uses a technique called Neuroevolution of Augmenting Topologies (NEAT) to train AI agents directly from raw pixel input, mimicking how a human player perceives the game. This innovative method bypasses the need for predefined behaviors or complex game-state information, making it particularly suitable for games without official programming interfaces.
The core of this research involves NEAT, an evolutionary algorithm that optimizes neural networks by evolving both their structure and weights. Starting with simple network architectures, NEAT incrementally increases complexity over generations, allowing the AI to adapt and grow in response to the game’s challenges. This is a significant advantage in environments like Dark Souls, where optimal network structure isn’t known beforehand and rewards can be sparse or delayed.
To facilitate this vision-based approach, the researchers developed the Dark Souls API (DSAPI), a novel Python framework. DSAPI leverages real-time computer vision techniques to extract crucial game metrics, such as player and enemy health, directly from the screen. It also handles menu navigation and game resets, streamlining the training and evaluation process for the AI agents. This framework allows the AI to interact with the game in a non-intrusive way, processing screen captures to gather necessary gameplay data.
The experiments focused on training agents to defeat the Asylum Demon, the first boss in Dark Souls. This specific scenario was chosen because it simplifies the broader complexities of the game, isolating combat strategy as the primary learning objective. The neural network received downscaled visual input (40×30 pixels) and outputted a discrete set of control actions, including movements, attacks, and defensive maneuvers. The agents’ performance was evaluated using a fitness function based on player and boss health, rewarding survival and offensive effectiveness.
The results of the study are promising. The NEAT-evolved agents, trained solely on visual input, achieved a maximum success rate of approximately 35% against the Asylum Demon. This demonstrates the viability of neuroevolution in tackling complex, visually intricate gameplay scenarios. The analysis of the evolved neural networks revealed the emergence of sophisticated strategies, such as reactive healing behaviors and opportunistic attacking. This work represents an interesting application of vision-based neuroevolution, highlighting its potential use in a wide range of challenging game environments lacking direct API support or well-defined state representations. For more details, you can read the full paper here.
Also Read:
- Self-Evolving Neural Networks: Adapting Architecture with Monte Carlo Tree Search
- Guiding Digital Life: A New Era of Human-AI Collaboration Through Language
Future work in this area includes exploring more advanced image processing techniques, incorporating additional gameplay actions, and extending the evaluation scenarios to include environmental navigation and more complex enemy interactions. Researchers also plan to investigate alternative neuroevolutionary methods and hybrid approaches combining reinforcement learning to further enhance agent performance and adaptability.
