TLDR: Polymath is a new self-optimizing AI agent that tackles complex, dynamic problems without relying on labeled datasets. It uses a two-tiered approach: a task flow graph to break down problems into subtasks, and code-represented workflows for executing those subtasks. A novel hierarchical optimization methodology, combining graph optimization and a self-reflection-guided evolutionary algorithm, allows Polymath to continuously refine its strategies. This approach has shown an 8.1% average performance improvement over state-of-the-art methods across coding, math, and multi-turn QA benchmarks, and demonstrated strong results in a real-world industrial application.
Large Language Models (LLMs) have shown impressive abilities in various fields, from generating code to making complex decisions. However, their effectiveness in solving real-world problems often depends on carefully designed, human-engineered workflows. These workflows, like Chain-of-Thought or ReACT, are typically built manually for specific tasks, which makes them difficult to scale and adapt to new challenges.
Many recent efforts have focused on automating the creation and optimization of these agentic workflows, often using code-based representations. Yet, a common limitation is their reliance on labeled datasets for training and optimization. This makes them less effective for dynamic, real-world problems where such data is unavailable or constantly changing.
Introducing Polymath: A Self-Optimizing Agent
To address these challenges, researchers from Nvidia Research and Nvidia have introduced Polymath, a novel self-optimizing agent. Polymath features a dynamic hierarchical workflow that combines the flexibility of task flow graphs with the expressiveness of code-represented workflows. This allows it to tackle a wide array of real-world, dynamic problems without needing pre-labeled data.
The core of Polymath’s innovation lies in its unique optimization methodology. It integrates a multi-grid-inspired graph optimization technique with a self-reflection-guided evolutionary algorithm. This allows Polymath to refine its workflows on the fly, learning and improving as it goes.
How Polymath Works
Polymath operates on two main levels:
- Task Flow Graph: At the higher level, Polymath uses a task flow graph to break down complex problems into smaller, more manageable subtasks. An LLM-based task flow planner monitors the execution of these subtasks, deciding whether to proceed, rerun a subtask, or apply ‘jump logic’ based on the results. This divide-and-conquer approach ensures that even intricate problems can be systematically addressed.
- Code-Represented Subtask Workflow: Each individual subtask within the graph is handled by a code-based workflow. These workflows are built by combining various LLM assistants, such as a coding assistant, a reasoning assistant, or a file reader. This code-based representation ensures stable and robust execution, minimizing issues like hallucinations often seen in LLMs.
The optimization process is also hierarchical. The task flow graph itself is optimized using a multi-grid-inspired method that balances the complexity and success rate of subtasks. For the code-represented workflows, a self-reflection-guided evolutionary algorithm continuously enhances them. This algorithm uses feedback from LLM judges, which provide multi-objective scores (InstructionFollowing, Correctness, MatchHighLevelPlanProgress, and a Combined score) and self-reflections, to iteratively improve the code without needing external labeled datasets.
Also Read:
- Enhancing Mathematical Problem-Solving with Knowledge Graphs and Executable Code
- MOTIF: Advancing Algorithmic Design Through Competitive LLM Interaction
Performance and Real-World Impact
Polymath has been extensively tested on six benchmark datasets covering coding, mathematics, and multi-turn question answering tasks. The results are impressive: Polymath achieved an 8.1% average improvement over state-of-the-art baselines. Notably, it outperformed AFlow, another automated workflow optimization method, by an average of 21.2% on MATH lv5* benchmarks.
Beyond standard benchmarks, Polymath also demonstrated its capability in a real-world industrial case study in hardware design. This challenging problem involved processing multiple files, block diagrams, and a 100-page datasheet. Polymath achieved a 14.4% higher accuracy compared to existing agentic flows, highlighting its effectiveness and adaptability in complex, practical scenarios.
In essence, Polymath represents a significant step forward in creating more autonomous and adaptable AI agents. By dynamically optimizing its problem-solving strategies without relying on labeled data, it offers a flexible and powerful solution for a wide range of dynamic, real-world problems.
For more in-depth details, you can read the full research paper here.
