TLDR: This research paper proposes a new architecture for data systems to efficiently support Large Language Model (LLM) agents, which are expected to become the dominant workload. It introduces ‘agentic speculation’—high-throughput, exploratory querying—and identifies its characteristics: scale, heterogeneity, redundancy, and steerability. The paper outlines how data systems need to evolve with new query interfaces (using natural language ‘briefs’), advanced probe optimization (to ‘satisfice’ and share computation), and a redesigned storage layer including an ‘agentic memory store’ and efficient ‘branched updates’ to handle the unique demands of AI agents.
The rise of Large Language Model (LLM) agents is poised to fundamentally change how we interact with data systems. A recent research paper, “Supporting Our AI Overlords: Redesigning Data Systems to be Agent-First,” by Shu Liu, Soujanya Ponnapalli, Shreya Shankar, Sepanta Zeighami, Alan Zhu, and a team of distinguished researchers from UC Berkeley, argues that current data systems are not equipped to handle these new agentic workloads and proposes a new architecture to support them more effectively. The full paper can be found here.
LLM agents are designed to act on behalf of users, manipulating and analyzing data with unprecedented efficiency. Unlike human users, these agents engage in what the researchers call “agentic speculation”—a high-throughput process of exploring data and formulating solutions. This involves issuing hundreds or even thousands of requests per second, many of which are exploratory, aimed at discovering metadata like table schemas or column statistics, or testing partial solutions.
This speculative behavior presents significant challenges for today’s data systems. Traditional systems are built for intermittent, targeted queries from humans or applications, not for the sheer volume and often redundant nature of agentic exploration. The paper highlights four key characteristics of agentic speculation that offer opportunities for redesign: scale, heterogeneity, redundancy, and steerability.
Understanding Agentic Speculation
The researchers conducted case studies to illustrate these characteristics. Using the BIRD text2SQL benchmark, they found that both parallel and sequential agentic speculation significantly improve accuracy, with success rates increasing by 14%–70% as the number of requests grows. This demonstrates the immense scale of queries agents can generate for a single task.
Furthermore, their analysis revealed substantial redundancy in agentic requests. Across many queries, the number of distinct sub-plans or sub-expressions was often a small fraction (less than 10-20%) of the total, indicating a vast potential for sharing computation and eliminating redundant work. This is a crucial insight for optimizing agent-first systems.
Another case study, involving a data agent combining information from two different backend databases, showed the heterogeneous nature of agentic information needs. Agents typically start with coarse-grained metadata exploration (e.g., table schemas) before moving to more specific column statistics and then formulating partial or complete queries. These phases often overlap, but the early stages are heavily focused on discovery.
Finally, the studies demonstrated that agentic speculation is highly steerable. By providing “grounding hints” (background information pertinent to the task), the number of queries agents needed to complete tasks could be reduced by over 20%. This suggests that proactive feedback from data systems can significantly improve agent efficiency.
A New Architecture for Agent-First Data Systems
Based on these insights, the paper proposes a new architecture for agent-first data systems. At its core, this architecture introduces “probes” instead of traditional queries. These probes go beyond SQL, allowing agents to specify their goals, current phase (metadata exploration or solution formulation), approximation needs, and priorities in natural language “briefs.” An “agentic interpreter” within the database parses these probes.
The system would also feature a “probe optimizer” designed to “satisfice” – producing reasonable results that address needs without necessarily evaluating queries completely. This optimizer would leverage multi-query optimization and approximate query processing, but with new objectives tailored to agentic workloads, such as minimizing total time spent on an agent’s probes rather than just overall throughput.
To support the dynamic and redundant nature of agentic workloads, the storage and transactional components also need to evolve. The researchers propose an “agentic memory store” to act as a semantic cache, storing grounding information gleaned from prior probes and metadata. This store would help agents quickly find relevant information and avoid repeated queries. For updates, a “shared transaction manager” would efficiently handle the redundancy in state across many potential transactions, supporting “branched updates” where agents explore multiple “what-if” hypotheses in parallel.
Also Read:
- Agentic Reinforcement Learning: Empowering LLMs as Autonomous Decision-Makers
- Smart Planning for LLM Agents: Balancing Speed and Expense
Rethinking Query Interfaces, Processing, and Storage
The paper delves into specific research opportunities across different layers. For query interfaces, it suggests moving beyond SQL to allow agents to provide natural language briefs, specify termination criteria for queries, and use new semantic similarity operators to find relevant data or metadata when they lack knowledge of data organization.
Data systems should also proactively steer agents by providing auxiliary data-centric information and feedback on efficiency and costs. This could involve “sleeper agents” within the system identifying related tables, explaining why a query returned an empty result, or suggesting ways to modify probes for better performance.
In terms of processing, the probe optimizer would need to decide not only how to execute queries but also which queries to execute and to what degree of approximation, balancing cost and accuracy. It would exploit redundancy through multi-query optimization and incremental processing, while also considering inter-probe optimizations across turns and agents to minimize future follow-up probes.
Finally, the storage layer needs to adapt to dynamic workloads and the need for cooperative state sharing. The agentic memory store would store artifacts like prior probe results and metadata, supporting semantic search. For transactions, the paper advocates for “multi-world isolation” to handle branched updates efficiently, allowing thousands of near-identical snapshots and ultra-fast rollbacks, moving beyond traditional linear execution models.
In conclusion, the paper presents a compelling vision for redesigning data systems to natively support the unique demands of LLM agents. By embracing the characteristics of agentic speculation—scale, heterogeneity, redundancy, and steerability—researchers can unlock significant productivity gains, transforming how we interact with and manage data in the age of AI.
