TLDR: A distributed evolutionary simulation of prey and predator agents revealed that social reward for observing the same species is more crucial for prey survival than fear. Fear-like negative rewards for observing predators only evolved after social rewards were established. The study also found that predator hunting ability influences fear emergence, and stationary threats can lead to unexpected positive associations due to food accumulation, preventing fear evolution.
Fear is a fundamental brain function, crucial for detecting danger and learning to avoid threats. For a long time, it has been widely believed that fear evolved primarily under the pressure of predators. However, experimentally reproducing this evolutionary process in real-world settings is incredibly challenging.
To delve deeper into the intricate relationship between environmental conditions, the evolution of fear, and the development of other vital rewards like food and social connection, researchers Yuji Kanagawa and Kenji Doya from the Okinawa Institute of Science and Technology Graduate University developed a sophisticated distributed evolutionary simulation. This simulation allowed prey and predator agents to co-evolve their innate reward functions, including a potential fear-like response to predators, while simultaneously learning behaviors through reinforcement learning.
The findings from this simulation were quite surprising and shed new light on the evolutionary path of fear. Contrary to the initial assumption that fear would be the primary driver of survival, the simulation revealed that social reward – the positive feeling associated with observing and interacting with the same species – played a more critical role in the prey’s ability to survive and thrive. In fact, the fear-like negative reward for observing predators only began to evolve in prey after they had already acquired this crucial social reward.
The study also explored how different characteristics of predators influenced the evolution of fear. It was found that predators with increased hunting ability, simulated by a larger ‘mouth’ size, amplified the emergence of fear in prey. However, fear evolution proved to be more stable when predators were non-evolving and less effective at chasing prey, making avoidance easier for the prey.
Another fascinating aspect of the research involved examining environments with stationary threats, such as harmful pitfalls, instead of moving predators. Surprisingly, fear did not evolve in response to these fixed dangers. Instead, prey agents developed positive rewards for being near these pitfalls. This counter-intuitive result occurred because areas around stationary threats often accumulated abundant, uneaten food, making these seemingly dangerous zones beneficial for foraging. This suggests that the spatial relationship between threats and food sources significantly impacts how fear evolves.
The simulation model extended previous work on food reward evolution, incorporating a prey-predator dynamic. Both prey and predators were modeled as circles in a 2D physics environment, equipped with proximity and tactile sensors. They consumed energy, reproduced based on energy levels, and died if energy dropped too low or due to age. Their reward functions, which included terms for food intake, motor output, and observing other agents (prey or predators), were genetic parameters that evolved over generations with mutations. Agents learned to maximize these rewards using a reinforcement learning algorithm called Proximal Policy Optimization (PPO).
Also Read:
- Balancing Exploration: How AI Agents Learn from Curiosity and Control
- Advancing Multi-Agent Intelligence with Generative AI
In summary, this groundbreaking research suggests a complex interplay between fear and social reward throughout evolution, alongside the specific nature of predators and other environmental threats. It highlights that in environments where grouping offers protection, social rewards may evolve first, paving the way for the subsequent development of fear. For more details, you can read the full research paper here.
