TLDR: Google has launched its Gemini 2.5 Computer Use model, a specialized AI built on Gemini 2.5 Pro, enabling AI agents to interact with graphical user interfaces (GUIs) like humans. This model automates complex digital tasks such as web navigation, form filling, and UI testing, demonstrating superior performance against competitors.
Google has officially unveiled its Gemini 2.5 Computer Use model, a significant advancement in artificial intelligence designed to empower AI agents with the ability to interact directly with graphical user interfaces (GUIs) in a human-like manner. Announced on October 8, 2025, this specialized AI model, built upon the robust capabilities of Gemini 2.5 Pro, is set to revolutionize web automation and UI testing.
The Gemini 2.5 Computer Use model allows AI agents to perform a wide array of digital tasks that traditionally require human intervention. This includes navigating complex web pages, filling out forms, clicking buttons, typing text, scrolling through content, and manipulating interactive elements such as dropdowns and filters. This capability extends beyond structured APIs, addressing the need for AI to operate within interfaces designed for human interaction.
At its core, the model operates through a continuous loop. Developers provide the AI agent with a user request, a current screenshot of the interface, and a history of recent actions. The model then analyzes these inputs, generating a suggested action in the form of a function call (e.g., `click_at`, `type_text_at`). Client-side code executes this action, a new screenshot is captured, and the loop repeats until the task is completed or a safety rule is triggered. The system supports 13 predefined UI actions, including `open_web_browser`, `wait_5_seconds`, `go_back`, `go_forward`, `search`, `navigate`, `click_at`, `hover_at`, `type_text_at`, `key_combination`, `scroll_document`, `scroll_at`, and `drag_and_drop`. Custom functions can also be integrated for non-browser surfaces.
Google states that the Gemini 2.5 Computer Use model demonstrates strong performance on multiple web and mobile control benchmarks. In tests conducted by browser automation company Browserbase, the model achieved high accuracy on browser control tasks while maintaining lower latency compared to competing models. Google’s internal payments team has also utilized the model to fix broken UI tests, successfully recovering over 60% of failed test runs, highlighting its practical efficacy.
The model is primarily optimized for web browsers and shows strong performance on mobile UI control tasks. However, it is not yet optimized for controlling desktop operating systems. A crucial aspect of its design is a built-in safety monitor that can block prohibited actions or require user confirmation for ‘high-stakes’ operations like payments or accessing sensitive records.
Also Read:
- Google’s AI Search Expands Globally, Enhancing Accessibility with Arabic and Dozens More Languages
- Google DeepMind Unveils CodeMender: AI Agent Automates Software Vulnerability Remediation
The Gemini 2.5 Computer Use model is currently available in public preview through the Gemini API on Google AI Studio and Vertex AI, offering developers the tools to build sophisticated AI agents capable of seamless interaction with digital environments.
