TLDR: FELA (Feature Engineering LLM Agents) is a novel multi-agent evolutionary system that automates feature engineering for complex industrial event log data. It leverages specialized LLM agents (Idea, Code, Critic, Evaluation) to collaboratively generate, validate, and implement features, guided by an insight-driven self-evolution paradigm. FELA integrates reinforcement learning and genetic algorithms with a hierarchical knowledge base and dual-memory system to balance exploration and exploitation. Experiments show FELA generates explainable, high-performing features, significantly improving model performance on real-world datasets and reducing manual effort.
In the world of digital services, event log data, which captures detailed user actions and system events, is an incredibly valuable resource. However, this data is often vast, complex, and diverse, making the process of extracting meaningful insights—known as feature engineering—a significant challenge. Traditional automated methods often fall short due to their limited ability to explain their results, rigid operations, and poor adaptability to complex data types.
Addressing these challenges, a new system called FELA (Feature Engineering LLM Agents) has been proposed. FELA is a multi-agent evolutionary system designed to automatically create useful and high-performing features from intricate industrial event log data. It cleverly combines the reasoning and coding abilities of large language models (LLMs) with a self-evolution process guided by insights.
FELA operates with a team of specialized agents: Idea Agents, Code Agents, and Critic Agents. These agents work together to generate, validate, and implement new feature ideas. An Evaluation Agent then gathers feedback and updates a structured knowledge base and a dual-memory system, ensuring continuous improvement. Furthermore, FELA incorporates an agentic evolution algorithm that blends reinforcement learning and genetic algorithm principles. This helps the system explore new ideas while also making the most of promising ones.
The system’s architecture is built around three core components. First, a multi-agent collaborative architecture uses LLM-based agents with specific roles. Idea agents propose new feature hypotheses based on data semantics, while Code agents translate these ideas into executable code. Critic agents are crucial for verifying the outputs of both idea and code agents, mitigating issues like ‘hallucination’ common in LLMs and improving reliability.
Second, FELA features an explainable and steerable knowledge structure. It organizes knowledge hierarchically into ‘ideas’ (high-level, human-readable insights) and ‘features’ (their logical and mathematical realizations). This structure allows for transparency and control over the evolution process, enabling users to guide the exploration. It also makes it easy to integrate new ideas from experts or external sources, fostering human-machine collaboration.
Third, an agentic evolution algorithm is introduced, combining genetic algorithms and reinforcement learning. Ideas and features evolve in an ‘island-based’ population model, and a multi-armed bandit mechanism (using Upper Confidence Bound or UCB) dynamically balances exploring new ideas and exploiting successful ones. An Evaluation Agent summarizes past experiences to update a dual-memory system: a short-term memory for local context and a long-term memory for broader knowledge transfer. This helps FELA learn from experience and avoid inefficient searching.
Extensive experiments on real-world industrial datasets have shown that FELA can generate features that are not only explainable and relevant to the specific domain but also significantly improve the performance of predictive models. This reduces the need for extensive manual effort from data scientists. The results highlight the significant potential of LLM-based multi-agent systems as a versatile framework for automated, understandable, and adaptable feature engineering in complex real-world settings.
The paper, titled “FELA: A Multi-Agent Evolutionary System for Feature Engineering of Industrial Event Log Data,” was authored by Kun Ouyang from LIGHTSPEED STUDIOS, Tencent, and the National University of Singapore; Haoyu Wang from the Department of Electronic Engineering, Tsinghua University, and LIGHTSPEED STUDIOS, Tencent; and Dong Fang from LIGHTSPEED STUDIOS, Tencent. You can read the full research paper here.
Also Read:
- FM Agent: A New AI Framework for Autonomous Scientific Discovery and Optimization
- Unpacking Anomaly Detection: Why Simplicity Outperforms Complexity in Industrial Time Series
FELA’s ability to handle complex data, provide explainable results, and continuously evolve makes it a powerful tool for modern enterprises dealing with massive and heterogeneous event logs. It represents a significant step forward in automating a traditionally labor-intensive and challenging aspect of data science, promising improved model performance and reduced operational costs across various business scenarios.
