Windows 11’s “Hey, Copilot” Brings Screen-Aware Voice and Vision to Your PC

Microsoft is rolling out an upgraded Copilot experience for Windows 11 that lets you say “Hey, Copilot” and ask questions about what’s on your screen. The update combines Copilot Voice and Copilot Vision so you can query images or app content — for example, asking where a vacation photo was taken, plotting flights, or getting budgeting tips.

Unlike past attempts at desktop voice assistants, this Copilot rollout ties conversational AI directly to what Windows sees on your screen. Short promotional demos show practical use cases: locating settings in apps like Spotify, writing short biographies from photo portfolios, and suggesting next steps without manual searching.

• Key features: “Hey, Copilot” voice activation, Copilot Vision (screen analysis), and Copilot Actions (automating desktop tasks).
• Privacy & data: These features are cloud-based; image and screen data are sent to Microsoft’s servers. Microsoft says the features are opt-in and provides permission controls and runtime isolation for Copilot Actions.
• Copilot Actions: Designed to perform tasks (resizing photos, filling forms, interacting with websites) in the background. Actions are visible step-by-step in the Copilot app and can be interrupted by the user.
• Availability: “Hey, Copilot” and Copilot Vision are rolling out now to Windows 11 PCs with Copilot access; Copilot Actions and the Ask Copilot taskbar feature will be introduced gradually to Windows Insiders.

Microsoft positions Copilot as a step toward a more conversational, context-aware computing experience — akin to a Star Trek-style ship computer. But the cloud-based nature of screen analysis revives familiar trust and privacy concerns, especially after earlier missteps with other AI features.

For more details, see Microsoft’s Copilot overview (Microsoft Copilot) and the original coverage at Engadget.

Discussion: Will you enable “Hey, Copilot” or let Copilot Actions run tasks on your PC — are the convenience gains worth the privacy trade-offs?