TLDR: FLEX (Forward Learning with Experience) is a novel, gradient-free learning paradigm that allows Large Language Model (LLM) agents to continuously improve and adapt through accumulated experience. Instead of modifying model parameters, FLEX builds and refines a structured ‘experience library’ from successes and failures. This approach demonstrates significant performance gains in mathematical reasoning, chemical retrosynthesis, and protein fitness prediction. Key findings include a predictable scaling law of performance with experience accumulation and the ability to transfer learned experience between different agents, fostering a new era of self-evolving and transparent AI.
Large Language Models (LLMs) have brought about significant advancements in reasoning and problem-solving, powering autonomous agents across various domains. However, a fundamental limitation of these agents has been their static nature; once trained, their parameters are frozen, preventing them from learning and evolving from new experiences during deployment. This contrasts sharply with how intelligent beings continuously grow and adapt.
A new learning paradigm called Forward Learning with Experience (FLEX) addresses this challenge. FLEX enables LLM agents to continuously evolve through accumulated experience without relying on traditional gradient-based learning methods. This approach is designed to be gradient-free, meaning it doesn’t require computationally expensive back-propagation or direct modification of the LLM’s internal parameters, which is particularly beneficial for closed-source models.
How FLEX Works
At its core, FLEX cultivates scalable and inheritable evolution by building a structured ‘experience library’. This library is populated through continuous reflection on both successes and failures encountered by the agent while interacting with its environment. Instead of fine-tuning model parameters, FLEX focuses on constructing and leveraging this evolvable experience library to guide future reasoning.
The FLEX paradigm operates through a forward learning loop with two main phases: first, the agent extensively explores to gather and distill new problem-solving experiences, which are then integrated into the library. Second, the agent utilizes the most relevant experiences from this library to guide its reasoning on new tasks. As the experience library grows, it accumulates diverse and high-quality problem-solving strategies, allowing agents to leverage prior knowledge for increasingly superior performance.
Key Advantages and Properties
FLEX offers several profound properties for continuous evolution. It evolves a persistent experience library that continuously aggregates cross-task experiences, ensuring knowledge retention. The library can continuously expand and refine, allowing agent performance to scale with accumulated knowledge. Furthermore, FLEX stores strategies distilled from experiences in a semantic form, making the library both transferable and interpretable. This design enables seamless inheritance across different agents, avoiding redundant and costly re-training.
Demonstrated Effectiveness
The effectiveness of FLEX has been validated through extensive experiments across challenging scientific domains. It delivered substantial improvements in mathematical reasoning (up to 23% on AIME25), chemical retrosynthesis (10% on USPTO50k), and protein fitness prediction (14% on ProteinGym). These results highlight FLEX’s ability to significantly enhance reasoning capabilities and knowledge utilization across diverse tasks.
Scaling and Inheritance of Experience
A significant finding from the research is the identification of a clear scaling law of experiential growth. This means that an agent’s performance scales predictably with the size of its accumulated experience library. As the library grows, both training and test accuracy show consistent improvement, indicating that accumulated knowledge stabilizes predictions and enhances reasoning robustness.
Perhaps even more impactful is the phenomenon of experience inheritance. The experience library acts as a lightweight, plug-and-play module that can be seamlessly transferred between different agents. This allows for the distillation of expertise from strong models to uplift weaker ones, and, remarkably, even enables experiences from weaker models to provide substantial benefits to stronger ones. This suggests that the learned experience captures fundamental, high-level strategies that are model-agnostic, paving the way for a universal experience library that can improve a wide ecosystem of agents.
Also Read:
- SofT-GRPO: Advancing LLM Reasoning with Gumbel-Reparameterized Soft-Thinking
- Enhancing AI’s Math Skills: A Self-Evolving Approach to Multimodal Reasoning
Towards a New Era of AI
FLEX represents a significant step towards creating intelligent agents that can continuously learn and evolve from their interactions, much like living organisms. By redefining learning as a forward exploration and experience distillation process, it offers a pathway to more dynamic, collaborative, and transparent AI systems. This research illuminates a promising direction for artificial intelligence, moving towards open-ended evolution and collective wisdom. For more details, you can refer to the original research paper: FLEX: Continuous Agent Evolution via Forward Learning from Experience.
