TLDR: This paper introduces a platform combining reinforcement learning (RL) with language models (LMs) to study how narrative frameworks influence AI decision-making. Using a dual-system architecture in a gridworld environment, the research found that narrative-guided agents, particularly those inspired by a Westworld AI, demonstrated improved efficiency and success rates compared to pure RL or basic instruction-following agents. The platform provides a foundation for exploring the synergy between optimization-based learning and symbolic reasoning.
Artificial intelligence systems are becoming increasingly sophisticated, capable of both making complex decisions and engaging in advanced narrative reasoning. However, these two powerful capabilities have largely been studied in isolation. A new preliminary experimental platform aims to bridge this gap by combining reinforcement learning (RL) with language model (LM) reasoning to explore how narrative elements can influence AI decision-making.
The research introduces a dual-system architecture designed to examine how narrative frameworks could shape reward-based learning in AI. This innovative setup features a reinforcement learning policy that suggests actions based on past experiences, and a language model that processes these suggestions through various narrative frameworks to guide the final decisions. This modular design allows for controlled experimentation with different environmental complexities, narrative parameters, and the interaction between RL and narrative-based choices.
The platform was implemented in a configurable gridworld environment, where AI agents navigate from a starting point to a goal while avoiding obstacles. The RL component uses basic Q-learning, focusing on short training periods to highlight the potential influence of narrative. For narrative processing, the study primarily utilized GPT-4o-mini, balancing performance with computational efficiency.
Initial experiments explored three aspects of narrative influence. First, a baseline was established by comparing pure RL agents against a basic LLM+RL implementation using direct navigation instructions. Second, various grid sizes and obstacle densities were tested to identify conditions where narrative guidance might have a significant impact. Finally, several distinct narrative frameworks were introduced: a Theseus-inspired labyrinth navigator, a Sherlock Holmes detective scenario, and a Westworld-inspired AI agent, all while maintaining consistent environmental parameters.
The findings revealed interesting patterns. While the addition of language model processing introduced computational overhead, the LLM+RL agents demonstrated more efficient path-finding per interaction step. This efficiency was particularly noticeable in more complex environments (7×7 grids and larger), where LLM+RL agents achieved performance comparable to RL-only agents after significantly fewer training episodes.
When comparing the narrative frameworks, systematic differences in performance emerged. Narrative-driven agents consistently matched or exceeded the baseline LLM+RL agent (operating with simple instructions) in terms of episode success rate. However, they showed marked differences in path efficiency. The Theseus and Sherlock Holmes frameworks demonstrated comparable success rates to the baseline but with reduced average steps to completion. Most notably, the Westworld-inspired AI agent framework consistently outperformed other narratives across both higher success rates and lower average steps to completion.
Analysis of the agent behavior logs indicated that narrative frameworks influenced not just performance metrics but also the underlying decision-making patterns. The Westworld AI framework, for instance, showed a more consistent integration of RL suggestions with environmental observations, which likely contributed to its superior performance. These behavioral differences persisted across various environmental configurations, suggesting a robust influence of narrative framing on decision-making strategies.
Also Read:
- PolicyEvolve: Advancing AI Strategies in Multi-Player Games with Evolving Programmatic Policies
- Enhancing Vision-Language Models Through Reinforcement Learning and Preference Optimization
This preliminary work demonstrates the basic feasibility of combining narrative frameworks with reinforcement learning in a controlled experimental platform. While the results are exploratory, they suggest that narrative context can indeed influence agent behavior, particularly in complex environments. This opens up promising avenues for future research, including evaluating performance across different language model scales, exploring more complex reasoning schemas like chain-of-thought prompting, and investigating more sophisticated narrative frameworks and their interaction with various reinforcement learning algorithms. The full research paper can be found here: Narrative-Guided Reinforcement Learning: A Platform for Studying Language Model Influence on Decision Making.
