Windows 11 Experiments with App Screen and File Sharing Using Copilot AI

Introducing Copilot on Windows 11

Microsoft’s Copilot on Windows 11 is currently being tested for a new feature that lets users share their screen with the AI assistant. This functionality is available to Windows Insiders in the United States, and while it may raise some privacy concerns, Microsoft emphasizes the potential benefits.

How Copilot Vision Works

With Copilot Vision, users can share their screens to receive assistance while using various applications. For example, if you’re having trouble navigating through an app, Copilot can visually guide you on where to click or find specific options.

This interaction allows the AI to analyze what’s on your screen, provide helpful insights, and respond to questions in real time. If you’re trying to locate a setting in the Settings app, Copilot can pinpoint where to find it as you share your display.

The precise mechanism through which Copilot Vision processes screen data remains somewhat ambiguous, but it’s essential to note that this feature is cloud-based, relying on Microsoft’s server capabilities rather than local AI models.

Screen Sharing Made Easy

For Windows Insiders eager to try this feature, simply click the glasses icon in the Copilot app. From there, you can select the browser or application whose screen you wish to share and ask Copilot for assistance. If you decide to stop sharing your screen, just click the Stop button or the ‘X’ in the Copilot interface.

Copilot File Search: A Smart Way to Find Files

Another exciting addition is the “File Search” feature. This allows users to locate specific files on their devices by asking Copilot questions in plain language, such as “Can you find my resume?” In response, Copilot will search your computer’s storage to retrieve the document, which could be named resume.docx.

Supported File Types

Microsoft has confirmed that Copilot can read various file formats, including:

• .docx (Word documents)
• .xlsx (Excel spreadsheets)
• .pptx (PowerPoint presentations)
• .txt (text files)
• .pdf (PDF files)
• .json (JSON files)

Current Status of Copilot Features

At this moment, both the screen sharing (Copilot Vision) and file search features are gradually becoming available to participants in the Windows Insider Program. While Microsoft has not shared detailed information regarding data processing or encryption methods, users can anticipate more clarity on privacy and security once these features officially launch.