TLDR: This study investigates how deep reinforcement learning (DRL) agents process time under a dual-task paradigm, similar to human multitasking. Using a simplified Overcooked environment, agents trained on a dual task (time production + number comparison) significantly overproduced time compared to those on a single time production task. This emergent behavior mirrors human timing interference. Preliminary neural analysis in LSTM layers showed complex oscillations but no clear dedicated timer, suggesting further research is needed to understand the underlying time-keeping mechanisms in AI.
Researchers have delved into the intriguing question of how artificial intelligence (AI) agents perceive and track time, especially when faced with multiple tasks simultaneously. This study, drawing parallels with human cognitive processes, explores how deep reinforcement learning (DRL) agents handle temporal decision-making under a dual-task paradigm, revealing behaviors strikingly similar to those observed in humans.
The core of the investigation revolves around a well-established finding in human timing research: our perception of time can be significantly affected when we are engaged in a concurrent cognitive task. This phenomenon, known as temporal interference, suggests that temporal and cognitive processes might share resources in the brain.
To explore this in AI, the researchers designed a simplified version of the popular Overcooked environment. They created two scenarios: a single-task (T) setup and a dual-task (T+N) setup. In both, the AI agent’s primary goal was a time production task – specifically, cooking and delivering soup. This involved picking up an onion, placing it in an oven, and waiting for a specific target duration before retrieving the cooked soup. The agent received a reward for each soup delivered, making accurate timing crucial for optimal performance.
The crucial difference lay in the dual-task (T+N) variation. Here, in addition to the soup-cooking task, the agent also had to perform a concurrent number comparison task for a brief period after placing the onion in the oven. During this time, numbers between 1 and 10 appeared, and the agent had to respond with a specific action based on whether the number was less than 5 or 5 or more, receiving an immediate reward for correct responses.
Two DRL agents were separately trained for these tasks across four different target durations (7, 8, 9, and 10 time steps). The behavioral analysis of these agents yielded fascinating results. The dual-task (T+N) agents consistently exhibited a significant overproduction of time compared to their single-task (T) counterparts. This means they waited longer than necessary before checking the oven, effectively overestimating the required cooking time. This emergent behavior in the DRL agents directly mirrors findings from human behavioral studies on time production with interference.
Interestingly, this overproduction of time was an emergent behavior; the agents were not explicitly programmed with a timer or biologically inspired neural structures to mimic human timing biases. This suggests that DRL agents can spontaneously develop complex cognitive behaviors under certain conditions.
The study also included a preliminary analysis of the neural dynamics within the agents, specifically focusing on the long short-term memory (LSTM) layers, which are known for capturing temporal dependencies. While complex oscillatory dynamics were observed in the LSTM hidden states, which reset at the start of each new trial, the analysis did not reveal clear evidence of a dedicated or intrinsic timer mechanism. For some target durations, the single-task agent showed oscillations suggesting accurate timing, while the dual-task agent showed oscillations indicative of overproduction. However, a definitive link between these neural patterns and a specific time-keeping mechanism could not be established, indicating a need for further investigation into the underlying neural processes.
Also Read:
- Modeling Realistic Pedestrian-Driver Interactions with Human-Like Constraints
- Diagnosing AI’s Reasoning Abilities with TempoBench
This research represents a significant step towards understanding the parallels between emergent DRL behavior and behavior observed in biological systems. By demonstrating that AI agents can replicate human-like temporal interference effects in a dual-task paradigm, the study contributes to the growing field of interpreting cognition in deep learning models. While limitations such as the reward structure and the scope of neural analysis exist, this work opens new avenues for exploring the intricate mechanisms of time perception in both artificial and biological intelligence. You can read the full paper here.
