Microsoft Begins Testing Copilot Vision Update with Screen and App Recognition Features
Microsoft’s New Copilot Features for Windows
Microsoft has recently begun testing exciting new features for its Copilot app on Windows. These updates enhance the way users can interact with the AI assistant by enabling screen sharing and file searches, making the experience more intuitive and interactive.
What’s New in Copilot Vision?
Expanded Functionality
Initially available only in Microsoft’s Edge browser, Copilot Vision is now set to extend its capabilities to any app on your PC. This means you can share your screen or specific applications with the AI assistant, providing a more seamless way to get guidance. For instance, Copilot Vision can assist you with various tasks in applications like Adobe Photoshop, giving you step-by-step instructions on how to utilize its features effectively.
Enhancing User Experience
At a recent Microsoft event celebrating the company’s 50th anniversary, an early version of Copilot Vision was showcased. During this demonstration, the AI assistant helped a user navigate through a game of Minecraft and optimize settings in Microsoft’s Clipchamp video editing software. This feature aims to enhance user experience by offering tailored advice based on what you’re currently working on.
Features of Copilot Vision
Screen Sharing with Guidance
Users can share their screen or apps directly with Copilot, allowing the assistant to highlight areas on your screen to provide guidance. Although this feature is in development and may not yet be available in the beta version, its potential to enhance user interaction is significant.Analysis of Visual Content
Copilot Vision can analyze photos and webpages, offering suggestions and insights based on what is displayed. This capability will help users make more informed decisions while working with visual content.- Similarities to Microsoft Teams
Although it may remind users of Microsoft’s Recall feature, Copilot Vision is more akin to sharing your screen during a Microsoft Teams meeting. This similarity indicates that the AI can assist not only in static document handling but in real-time interactions as well.
File Search Capabilities
Alongside Copilot Vision, Microsoft is also testing a new file search feature within Copilot on Windows. This functionality allows users to inquire about the content of various file types stored on their PC. Supported formats include:
.docx(Word documents).xlsx(Excel spreadsheets).pptx(PowerPoint presentations).txt(Text files).pdf(PDF files).json(JSON data files)
With this feature, users can easily retrieve information about recently worked-on documents, improving overall productivity and workflow efficiency.
Accessibility Across Platforms
Both Copilot Vision and the file search capabilities just require the basic version of the Copilot app, meaning you don’t need a full Copilot Plus PC to access these features. Furthermore, these new functionalities are not limited to Windows; they can also be utilized on iOS and Android devices, which broadens their accessibility.
Testing Phase and Future Release
Currently, Microsoft is rolling out these innovative Copilot features to Windows Insiders, specifically limited to users in the United States during this testing phase. The company plans a wider release to all Windows 11 users within the upcoming weeks and months, enabling more users to benefit from these advanced AI capabilities.
By facilitating more intuitive user interactions with the software, these new updates position Microsoft Copilot as a potential game-changer for productivity across various digital tasks.
