TLDR: Salesforce AI Research has introduced WALT (Web Agents that Learn Tools), a new framework that enables large language model (LLM) agents to automatically identify and leverage reusable functionalities directly from any website. This approach aims to make web automation more robust and efficient by shifting from brittle, step-by-step UI interactions to deterministic tool invocations, significantly improving success rates and reducing reliance on extensive LLM reasoning.
Salesforce AI Research has announced the development of WALT (Web Agents that Learn Tools), an innovative framework designed to revolutionize web automation for large language model (LLM) agents. Traditional web agents often struggle with dynamic website layouts and complex, multi-step tasks, relying heavily on fragile, step-by-step UI interactions and extensive LLM reasoning. WALT addresses these limitations by enabling LLM agents to automatically discover and utilize website-native functionalities as reusable, invocable tools.
The core concept behind WALT is to reverse-engineer latent website functionality into robust, callable tools. Instead of an agent needing to reason through a long sequence of clicks and typing actions, it can simply call high-level operations such as `search(query)`, `filter(criteria)`, `sort(order)`, `post_comment(text)`, or `create_listing(details)`. This abstraction significantly reduces the computational burden on LLMs and increases the determinism and reliability of web automation tasks.
The WALT pipeline operates in two distinct phases: discovery and construction with validation. In the discovery phase, the framework systematically explores a website to propose potential tool candidates that align with common web goals, including content management, communication, and information discovery. Following this, the construction and validation phase converts these identified functionalities into deterministic scripts. This involves stabilizing selectors, promoting URLs where feasible, inducing an input schema for each tool, and rigorously validating the tool’s end-to-end functionality. This meticulous process ensures that as much work as possible is encapsulated within stable URL and form operations, reserving LLM-driven agentic grounding only for tasks that genuinely require flexible reasoning.
Initial evaluations of WALT have demonstrated significant performance improvements on established web agent benchmarks. On VisualWebArena, WALT achieved an average success rate of 52.9 percent, with specific results of 64.1 percent on Classifieds tasks, 53.4 percent on Shopping tasks, and 39.0 percent on Reddit tasks. These figures notably outperform existing baselines such as SGV (50.2 percent) and ExaCT (33.7 percent). Furthermore, on WebArena, WALT recorded an average success rate of 50.1 percent across various categories including GitLab, Map, Shopping, CMS, Reddit, and Multi-task scenarios. Beyond improved success rates, WALT also demonstrated a reduction in the number of actions required to complete tasks by approximately 21.3 percent, highlighting its efficiency.
Also Read:
- Salesforce Introduces MuleSoft Agent Fabric to Streamline and Govern Enterprise AI Agents
- Whatfix’s ‘Seek for Salesforce’ AI Agent Achieves Unprecedented Success in Trailhead Admin Challenges
This framework represents a crucial shift from sequence-based agents to functionality-grounded tools, promising a more robust and generalizable paradigm for browser automation. Salesforce AI Research plans to release a command-line interface (CLI) with `walt discover` and `walt agent` commands, alongside MCP serving for broader integration.
