TLDR: SciMaster introduces X-Master, a tool-augmented AI agent that uses code for flexible interaction with external tools. Its advanced ‘scattered-and-stacked’ workflow, X-Masters, has set a new world record on Humanity’s Last Exam (HLE) with a 32.1% score, surpassing OpenAI and Google. This open-source solution demonstrates a significant leap in general-purpose scientific AI capabilities, leveraging web search and parsing tools to enhance reasoning and problem-solving.
A new research initiative called SciMaster is making significant strides in developing advanced, general-purpose artificial intelligence agents for scientific discovery. The first part of this series introduces X-Master, a foundational AI agent designed to mimic how human researchers approach complex problems by flexibly using external tools during its thought process. This innovative approach, which views code as a language for interaction, allows X-Master to leverage various Python libraries and custom tools to enhance its reasoning capabilities.
The researchers further scaled the agent’s abilities through a sophisticated workflow named X-Masters. This system employs a ‘scattered-and-stacked’ approach, orchestrating multiple instances of the X-Master agent in specialized roles—Solver, Critic, Rewriter, and Selector—to systematically broaden and deepen its reasoning. The ‘scattered’ phase involves multiple Solver agents generating diverse solutions in parallel, with Critic agents refining them. The ‘stacked’ phase then synthesizes these outputs through Rewriter agents, leading to superior solutions, before a final Selector agent chooses the best answer.
A key benchmark for evaluating scientific AI agents is Humanity’s Last Exam (HLE), an exceptionally challenging test developed by nearly 1,000 subject experts globally. X-Masters has achieved a new state-of-the-art record on HLE, scoring an impressive 32.1%. This performance surpasses previous leading deep research products from major AI labs like OpenAI (26.6%) and Google DeepMind (26.9%), making X-Masters the first system to exceed the 30% threshold on this demanding exam. This achievement highlights the effectiveness of X-Master’s tool-augmented reasoning and the power of the inference-time agentic workflow.
The paper emphasizes that X-Master’s ability to interact with external environments, such as the internet, is crucial. It uses a ‘web search’ tool to find relevant webpages and a ‘web parse’ tool to extract detailed information, including content from scientific papers. This allows the agent to gather real-time, external knowledge, overcoming the limitations of its internal training data. The research also demonstrates X-Masters’ strong performance in specific scientific domains, such as biology, where it achieved state-of-the-art results on benchmarks like TRQA-lit.
Also Read:
- Unlocking Advanced AI Reasoning: A New Framework for Smarter Language Models
- MateInfoUB: A New Benchmark Reveals LLM Strengths and Weaknesses in Competitive Computer Science Education
Crucially, unlike many top-performing AI systems, X-Masters is an open-source solution. The researchers aim to share their valuable ‘know-how’ with the community, demonstrating that leading performance on challenging benchmarks is achievable with accessible open-source models without extensive retraining. This work provides a practical roadmap for enhancing large language model capabilities and lays the groundwork for future advancements in general-purpose scientific AI agents, with plans to develop specialized tools for literature analysis, scientific computing, and experimental workflows as part of the broader SciMaster series. You can read the full research paper here.
