TLDR: This research compares “action-aware” and “action-unaware” AI agents within the Active Inference framework. Action-aware agents know their past actions, while action-unaware agents must infer them from observations. The study shows that action-unaware agents can achieve comparable performance to action-aware ones in navigation tasks, despite a higher computational cost, suggesting a more biologically plausible model of cognition where explicit knowledge of one’s own actions isn’t necessary for effective planning and learning.
In the fascinating realm of artificial intelligence and cognitive science, researchers are constantly seeking to understand how intelligent agents perceive, learn, and act in complex environments. A prominent framework for this is Active Inference, which posits that adaptive agents operate by minimizing a quantity known as “free energy.” This approach offers a unified view of perception, learning, and decision-making as a process of approximate Bayesian inference.
At its heart, Active Inference involves two key types of free energy: variational free energy and expected free energy. Variational free energy helps agents understand their current situation and learn about their environment by accumulating evidence. Expected free energy, on the other hand, guides an agent’s future actions, helping it choose policies (sequences of actions) that are most desirable. This desirability is a balance between reaching preferred states (reducing “risk”) and gaining new, informative observations about the environment (reducing “ambiguity” and increasing “novelty”).
A central debate within the Active Inference community revolves around how agents account for their own actions. This paper introduces a crucial distinction between “action-aware” and “action-unaware” agents. Action-aware agents are like those with a perfect memory of their movements; they know exactly what actions they have taken in the past. This knowledge simplifies their planning, as they only need to consider future actions. In contrast, action-unaware agents operate without this explicit knowledge. They must infer their past movements from their observations, adding a layer of complexity to their decision-making process.
This difference mirrors a long-standing discussion in motor control, particularly concerning the presence or absence of an “efference copy” signal – a neural signal that informs the brain about impending movements. Traditional Active Inference formulations often align with the idea that biological agents, including humans, might not have direct access to such explicit motor information, making the action-unaware model potentially more biologically realistic.
Testing the Agents in Mazes
To compare these two types of agents, the researchers conducted experiments in two simulated navigation tasks: a T-maze and a 3×3 grid world. In the T-maze, a simpler environment, both action-aware and action-unaware agents successfully learned to find the optimal path to a goal state. While action-aware agents showed a slightly faster and more consistent learning curve, the action-unaware agents still achieved comparable performance relatively quickly, demonstrating their ability to adapt despite their inherent disadvantage.
The second experiment, a 3×3 grid world, presented a larger and more complex environment with multiple optimal paths to the goal. Again, both types of agents learned to navigate effectively. Interestingly, action-unaware agents exhibited more fluctuations in their “policy-conditioned free energy” – a measure related to how well a policy explains past observations. This suggests that action-unaware agents were exploring and assigning probabilities to various optimal paths, rather than settling on a single one, which is a natural consequence of their need to infer past actions.
Also Read:
- Guiding Multi-Agent Planning with Graph Neural Networks
- Landmarks Enhance Monte Carlo Planning for Uncertain Environments
Implications and Future Directions
The study’s findings are significant: action-unaware agents, despite lacking explicit knowledge of their past actions, can perform on par with action-aware agents. This supports the notion that sophisticated cognitive abilities can emerge even without direct access to internal motor commands, aligning with certain theories of biological motor control. However, this comparable performance comes at a computational cost. The perceptual inference stage for action-unaware agents is significantly more complex, requiring them to update many more probabilistic distributions to account for all possible past action sequences. This makes them less computationally efficient than their action-aware counterparts, especially in larger environments.
This research provides valuable insights into the mechanisms of active inference and its potential to model biological cognition. It highlights a trade-off between biological plausibility and computational efficiency. Future work may explore ways to make action-unaware agents more scalable, perhaps through methods like weight-based sampling of action sequences. For a deeper dive into the technical details, you can read the full research paper here: Active inference for action-unaware agents.
