TLDR: This research introduces a prototype that combines the planning flexibility of Deep Research systems (using LLM agents) with the efficient, optimized execution of semantic operators for AI-driven analytics over large unstructured datasets. By extending the Palimpzest framework with new ‘compute’ and ‘search’ operators and a ‘Context’ abstraction for dynamic data access, the system enables agents to write and execute optimized semantic operator programs. Evaluations show significant improvements in accuracy, cost, and runtime efficiency compared to existing approaches on complex analytical tasks.
The world of data analytics is rapidly evolving, driven by the incredible advancements in large language models (LLMs). These powerful AI tools are enabling new ways to analyze vast amounts of unstructured data, moving beyond traditional structured databases. However, building systems that can effectively perform these AI-driven analytics tasks comes with its own set of challenges.
Researchers have explored two main approaches: semantic operators and Deep Research systems. Semantic operators are like AI-powered versions of classic database operations (like filters, maps, joins) but specified in natural language. While they can be optimized for efficiency, they often struggle with interactive tasks or when data needs to be processed across many files simultaneously. Imagine trying to find a specific piece of information spread across hundreds of documents – a semantic operator might process each one individually, which can be slow and expensive.
On the other hand, Deep Research systems use one or more LLM agents to plan, execute, and refine answers to natural language questions over large datasets. These systems are highly flexible and can adapt their plans dynamically. However, their main weakness lies in execution; they don’t always optimize their query plans, leading to inefficiencies, “shortcuts,” or even premature termination before a task is fully completed. This means they might miss crucial information or take a very long time to get to an answer.
The core idea presented in the research paper, “Deep Research is the New Analytics System: Towards Building the Runtime for AI-Driven Analytics,” is to combine the best of both worlds. The authors, Matthew Russo and Tim Kraska from MIT, envision a new runtime for AI-driven analytics that merges the optimized execution capabilities of semantic operators with the flexibility and dynamic planning of Deep Research systems. This approach aims to create a system that is both smart in its planning and efficient in its execution.
A New Prototype for AI-Driven Analytics
As a first step towards this vision, the researchers built a prototype that extends the Palimpzest framework. This prototype introduces two key innovations:
- New Semantic Operators: They added two new operators,
computeandsearch, which are powered by LLM agents. Crucially, these agents are equipped with a tool that allows them to write and execute optimized semantic operator programs. This means the flexible AI agents can now leverage highly efficient data transformations. - The Context Abstraction: A new concept called “Context” was introduced. This goes beyond simply iterating over data. A Context can support various access methods, like indexing for quick lookups or vector-based search, and can also incorporate custom tools provided by users. Each Context also has a natural language description, helping agents understand and interact with the data more effectively.
To further enhance efficiency, the prototype also includes a ContextManager. This component caches descriptions of previously processed “materialized” Contexts. Similar to how databases reuse materialized views, this allows the system to leverage past query executions to optimize new queries, significantly reducing the high cost and latency often associated with LLM invocations.
Optimized Performance
The paper highlights that this combined approach offers significant opportunities for optimization. While some advanced logical optimizations (like query rewriting or merging operations) are planned for future work, the prototype already implements physical optimizations such as selecting the best LLM model for a task and reusing cached Contexts.
Also Read:
- The Future of Data: Redesigning Systems for LLM Agents
- Unifying Semantic Data Processing with SQL: Introducing SABER
Evaluation and Results
The prototype was evaluated on two distinct queries:
- Identity Theft Reports (Kramabench dataset): This query involved calculating the ratio of identity theft reports between two specific years from a dataset of 132 files. A handcrafted semantic operator program struggled with this, sometimes returning incorrect ratios due to issues with cross-file reasoning. A basic Deep Research agent also performed poorly, often failing to find the correct files. The prototype, however, achieved a significantly lower error rate (0.02% compared to 17% for semantic operators and 27.56% for the CodeAgent), demonstrating its ability to dynamically plan and execute optimized programs to find the correct information.
- Enron Email Filtering: This task required filtering a subset of 250 Enron emails for firsthand discussions of specific business transactions and extracting sender, subject, and summary. A standard Deep Research agent (CodeAgent) was fast and cheap but had low quality (poor recall) because it relied on simple heuristics and manual validation. Another agent (CodeAgent+) given unoptimized semantic operators as tools achieved high quality but was very expensive and slow due to inefficient operator usage. The prototype, by writing optimized Palimpzest programs, achieved near-perfect output quality while saving 76.8% on cost and 72.7% on runtime compared to CodeAgent+. This was thanks to Palimpzest’s execution engine avoiding redundant computation and using cheaper models where possible.
In essence, the research demonstrates that by allowing flexible Deep Research agents to write and execute optimized semantic operator programs, the system can achieve both high accuracy and efficiency on complex AI-driven analytics tasks. This work lays a foundational step towards a more unified and powerful runtime for analyzing large, unstructured data lakes. You can read the full research paper here.
