TLDR: This research explores how biological intelligence can be simulated using active inference, a theory of behavior. By modeling decision-making in a simulated Pong game, similar to the DishBrain experiment with living neurons, the study shows that agents using memory-based learning (Counterfactual Learning) significantly outperform planning-based approaches. The findings highlight the crucial role of memory in adaptive decision-making and offer a biologically plausible, explainable framework for understanding purposeful behavior in AI.
The rapid advancements in artificial intelligence (AI) highlight a crucial need to understand how autonomous agents make purposeful decisions, especially for developing safe and efficient systems. While artificial neural networks have been dominant, recent research is exploring biologically based systems, such as networks of living biological neurons, for their potential in power efficiency, data efficiency, and providing more explainable models.
This research introduces a framework based on active inference, a broad theory of behavior, to model how embodied agents make decisions. The study uses generative models informed by experiments to simulate decision-making in a game-play environment, specifically mirroring setups that use biological neurons, like the DishBrain system.
The DishBrain system involves culturing cortical neurons on silicon chips and integrating them into a simulated Pong game. In this setup, the neurons learn to control a paddle to hit a ball, demonstrating adaptive intelligence through sensory feedback. This work contributes to the field of synthetic biological intelligence (SBI), which leverages living neural systems as computational substrates.
A key challenge in this field has been the lack of theoretical frameworks to comprehensively model the intelligence observed in these biological systems. Traditional AI models, especially deep learning, often struggle to capture the adaptive, memory-driven, and embodied nature of biological decision-making. To address this, the paper proposes a generative model inspired by DishBrain experiments, using active inference as its theoretical foundation.
The generative model used in this study is based on Partially Observable Markov Decision Processes (POMDPs). This model helps the agent form expectations about observations and predict future outcomes to control them. The state space of the model is quite large, with 2432 states representing the ball’s x and y coordinates and the paddle’s y coordinate, reflecting the experimental setup. The action space includes ‘Up’, ‘Down’, and ‘Stay’.
Decision-Making Approaches Explored
The researchers explored different decision-making schemes within the active inference framework:
-
Classical Active Inference (AIF-1): This method involves minimizing the expected free energy of future observations. However, it faces computational challenges due to the vast number of possible action sequences, making it practical only for one-step planning in complex environments.
-
Dynamic Programming in Expected Free Energy (DPEFE): This approach uses dynamic programming principles to efficiently plan over longer time horizons, addressing the computational limitations of classical active inference.
-
Counterfactual Learning (CFL): This method doesn’t rely on explicit planning but instead learns a state-action mapping based on a ‘Risk’ parameter. The agent learns from past experiences, making it suitable for environments requiring spontaneous decision-making and proving to be data-efficient.
Also Read:
- Building Safer AGI: A New Framework Merges Active Inference with Language Models
- Enhancing AI Agents with Lifelong Learning: Introducing Memp’s Procedural Memory
Key Findings and Insights
The simulations demonstrated significant findings. Agents using the counterfactual learning (CFL) method, especially with a memory-augmented version (CFL-T), showed improved performance with a higher memory horizon. For instance, the CFL-4 agent outperformed biological agents (MCC and HCC groups) in game metrics like hits per rally, percentage of aces, and percentage of long rallies. This highlights the importance of memory in such systems, even though long-term memory in DishBrain-like systems is still under investigation for biological plausibility.
In contrast, planning-based agents (AIF-1 and DPEFE) did not show the same level of improvement. While DPEFE agents performed similarly across different planning horizons, suggesting that increased planning depth wasn’t significantly beneficial in the Pong game, the CFL-3 agent consistently outperformed all other groups. This indicates that memory-based decision-making might be more effective for real-time, interactive game environments like Pong compared to planning-based approaches.
The study also provided explainable insights by analyzing model parameters. For CFL agents, a significant drop in the ‘risk’ parameter and a decrease in the Normalized Total Entropy (NTE) of the CL vector (state-action mapping) were observed for high-performing agents like CFL-4. This indicates that these agents were becoming more confident and refining their goal-directed learning strategies. For planning-based agents (DP-5, AIF-1), the entropy of transition dynamics decreased, showing they were learning about the environment. However, the entropy of prior preference distribution increased, suggesting that in a game like Pong, where the goal is simply to defend the ball rather than prefer specific positions, the agent doesn’t learn a particular preference for ball or paddle positions.
This work successfully demonstrates the active inference framework’s effectiveness in modeling purposeful decision-making within synthetic biological intelligence. It offers a biologically plausible and interpretable model for how systems like biological neuronal networks can achieve continuous learning and real-time responsiveness. The findings emphasize the advantages of active inference in capturing the sample efficiency and biological plausibility often lacking in traditional machine learning paradigms, making it well-suited for understanding systems like DishBrain.
The researchers suggest future work should focus on the biological instantiation of memory mechanisms and dissecting the roles of different memory types and adaptive dynamics in both synthetic and biological agents. This could lead to innovative hybrid systems combining biological intelligence’s adaptive strengths with computational frameworks’ explanatory power. Ultimately, this work contributes to a more integrated understanding of purposeful behavior, pushing forward the development of AI that is not only intelligent but also transparent and grounded in biological principles. For more details, you can read the full research paper here.
