Microsoft’s Copilot Studio Now Capable of Controlling a Computer
The Copilot logo is displayed on a smartphone, set against the backdrop of the Microsoft logo.
Photo: Jonathan Raa/NurPhoto
Microsoft Introduces a New Feature in Copilot Studio
Microsoft has enhanced Copilot Studio by adding a “computer use” feature. This innovative tool enables corporate AI agents to replicate human actions on a computer. They can now click buttons, select items from menus, and fill in text fields just as an employee would do.
According to The Verge, this new capability provides unprecedented flexibility, allowing AI agents to interact with any website or application, even those without an API (Application Programming Interface).
How the New Feature Works
This feature is similar to what OpenAI’s Operator and Claude offer but allows for greater versatility. As explained by Charles Lamana, Corporate Vice President of Microsoft Business & Industry Copilot, the “computer use” function allows AI agents to perform tasks that previously required human intervention. If a task can be done by a user manually, it can now be executed by Copilot as well.
The system is built to recognize various interface elements, so it can press the necessary buttons, fill out forms, and continue functioning even if the design or layout of a website changes. This adaptability is key to the feature’s functionality.
Practical Applications of the “Computer Use” Feature
The capabilities of Copilot Studio can significantly streamline various business processes. Some practical examples include:
- Automating routine data entry in Customer Relationship Management (CRM) systems.
- Conducting market research efficiently.
- Processing invoices and other documentation.
An important aspect of the “computer use” feature is its ability to analyze changes made on a webpage. If a site or application is updated, the feature adjusts accordingly, ensuring that scripts continue to function without interruption.
Microsoft has previously attempted to enhance Copilot’s functionality. Earlier this year, they introduced the “Actions” feature in the consumer version, allowing users to book restaurants, buy tickets, and place orders. However, this feature was limited to specific partners. In contrast, agents in Copilot Studio can engage with any web or desktop application without the need for prior agreements.
Benefits for Businesses
The introduction of the “computer use” function equips businesses with a powerful tool to create AI agents that mimic human operations within applications and websites. This is especially beneficial in situations where API integration is challenging or unfeasible, as it helps eliminate manual tasks.
Recent reports also indicate that Microsoft is considering acquiring TikTok, showcasing its ongoing interest in expanding its digital landscape. Additionally, the company is prompting users with older devices that cannot support Windows 11 to dispose of them, signaling the impending end of support for Windows 10.
