Introducing Enhanced Computer Utilization in Microsoft Copilot Studio for UI Automation
Introducing Computer Use in Copilot Studio
Microsoft is excited to introduce a new feature called computer use in Copilot Studio. This innovative tool allows agents within Copilot Studio to directly interact with various websites and desktop applications. If you’re interested in trying out this feature, we’re currently offering limited access through a research preview.
Accelerating AI Innovations
The pace of artificial intelligence (AI) advancements has never been faster, and Microsoft Copilot Studio is leading the way by integrating top-notch AI technology into a platform designed to address business challenges effectively. Just last month, we added deep reasoning capabilities, support for model context protocol (MCP), and made agent flows generally accessible within Copilot Studio.
What is Computer Use?
With the introduction of computer use, agents can now perform actions like clicking buttons, navigating menus, and typing into fields on the screen. This means they can accomplish tasks even if there’s no API available for direct interaction. If a human user can navigate an application, then so can the agent.
Enhancing Efficiency with Computer Use
This feature automatically adapts to changes in applications and websites, allowing agents to troubleshoot issues in real-time, thus ensuring uninterrupted workflow. It’s also built on Copilot Studio’s robust security and governance frameworks, which help organizations stay compliant with industry standards and regulations.
With computer use, developers can create agents that can automate tasks across different user interfaces on desktop and browser applications like Microsoft Edge, Google Chrome, and Firefox. Additionally, since the feature operates on Microsoft-hosted infrastructure, organizations do not have to worry about servicing their own servers. Any enterprise data remains within Microsoft Cloud boundaries and is not utilized to train the Frontier model, streamlining setup, reducing maintenance, and lowering infrastructure expenses.
Creating Value through Automation Scenarios
Consider these valuable use cases where computer use can make a significant difference:
- Automated Data Entry: Companies needing to input large amounts of data into a central system can automate this process, which lowers the risk of human error and cuts down on manual workload.
- Market Research: Marketing teams can use this tool to gather market data from various online sources automatically, saving time and providing insightful analysis without manual intervention.
- Invoice Processing: Finance departments can streamline invoicing by automating the extraction of data from invoices, enabling quick and easy entry into accounting systems while minimizing errors.
Transforming Robotic Process Automation (RPA)
Agents using computer use are reshaping how robotic process automation (RPA) functions. They are built to overcome traditional RPA pitfalls, such as the instability of user interface elements, and can better handle complex, dynamic environments. Here’s how Copilot Studio’s computer use feature enhances RPA:
- Real-Time Adaptability: The tool continues to function seamlessly even if buttons or screens change, ensuring your workflow remains uninterrupted.
- User-Friendly Interface: Users can describe their tasks in everyday language, eliminating the need for coding. They can also test and refine their prompts with real-time video feedback showing the reasoning behind the automation steps.
- Intelligent Decision-Making: The agents observe what’s on the screen and can make smart decisions quickly, even in rapidly changing situations.
- Activity Transparency: Developers can track computer use activities and review captured screenshots along with the reasoning process whenever they wish.
Future Innovation with Copilot Studio
Copilot Studio serves as a comprehensive agent platform aimed at helping organizations achieve their operational and AI goals. The introduction of computer use is a step toward empowering you to streamline processes, enhance productivity, and cultivate innovation.
If you’re curious about the new computer use feature, we invite you to express your interest by registering here for participation. More information will be shared about this exciting development at Microsoft Build in May 2025—make sure to register to join us!
