TLDR: Microsoft has unveiled a new ‘Computer Use’ feature within Copilot Studio, enabling AI agents to autonomously interact with any application or website through their graphical user interface. This groundbreaking capability allows agents to mimic human actions like clicking, typing, and navigating, even without direct API access, significantly advancing task automation and AI capabilities across various business functions.
Microsoft is advancing the capabilities of artificial intelligence with the introduction of a new ‘Computer Use’ feature in its Copilot Studio, allowing AI agents to operate computers independently by interacting directly with graphical user interfaces (GUIs) of applications and websites. This development marks a significant step towards more autonomous AI systems, as announced in an early access research preview and further detailed at events like Microsoft Build 2025.
The ‘Computer Use’ functionality empowers AI agents to treat any website or desktop application as a tool, performing actions such as clicking buttons, selecting menus, and entering text into on-screen fields. This is particularly impactful because it enables automation even for systems where a direct API connection is not available, meaning ‘if a person can use the app, the agent can too.’ Agents created with this feature are designed to adapt to changes in applications and websites automatically, adjusting in real-time using built-in reasoning to resolve issues without interrupting workflows.
According to Charles Lamanna, an executive for Business & Industry Copilot at Microsoft, ‘With computer use, agents can now interact with any system that has a graphical user interface!’ This capability is built on Copilot Studio’s robust security measures and governance frameworks, ensuring compliance with organizational and industry standards while running on Microsoft-hosted infrastructure, keeping enterprise data within Microsoft Cloud boundaries.
The implications for businesses are vast. These AI agents can be deployed for a wide range of tasks, including automating data entry, conducting market research, processing invoices, and handling complex customer support inquiries. They offer enhanced Robotic Process Automation (RPA) that is smarter and more robust than traditional RPA, capable of reasoning, reacting, and making decisions in dynamic user interfaces. The platform also supports no-code UI programming, allowing users to describe tasks in natural language and observe real-time automation with reasoning traces and UI previews.
Copilot Studio itself is a low-code tool designed for building, customizing, and deploying AI-powered agents. It integrates with the Power Platform and allows both business users and professional developers to create agents that can function as standalone copilots, within Power Platform apps, or embedded in other applications like Microsoft Teams or websites. The platform leverages powerful AI-powered capabilities, including the latest advancements in Natural Language Understanding (NLU) models and the Azure OpenAI GPT model, to generate conversational responses and create topics from simple descriptions.
Also Read:
- Microsoft Notepad Integrates Generative AI for Enhanced Text Editing
- Databricks Unveils AI Agent for Automated Data-to-Spreadsheet Generation
This new feature, currently available in public preview, is part of Microsoft’s ongoing expansion of Copilot Studio’s capabilities, which also includes multi-agent orchestration, enabling AI agents to collaborate across different departments like HR, IT, and marketing for complex tasks such as employee onboarding. The ability for AI to autonomously interact with computer interfaces represents a significant leap in automation and intelligent assistance, promising to transform how businesses operate and scale their solutions.
