tech
Critics scoff after Microsoft warns AI feature can infect machines and pilfer data
Integration of Copilot Actions into Windows is off by default, but for how long?
TL;DR
- Microsoft's experimental Copilot Actions in Windows can infect devices and steal sensitive data.
- Copilot Actions are "experimental agentic features" designed for tasks like organizing files and scheduling meetings.
- Known defects in large language models (LLMs) like hallucinations and prompt injection pose security risks.
- Prompt injection allows hackers to embed malicious instructions that LLMs follow, potentially leading to data exfiltration or malware installation.
- Microsoft advises enabling Copilot Actions only if users understand the security implications, but does not detail required user experience or preventative measures.
- Security experts criticize Microsoft's warnings, comparing them to decades-old warnings about macros, which remain a vector for malware.
- IT admins will have control over enabling or disabling Copilot Actions at account and device levels.
- Detecting exploitation attacks targeting AI agents is difficult for users.
- Microsoft's stated goals for agentic features include non-repudiation, confidentiality, and user approval for data access.
- Critics argue that reliance on users reading and understanding security warnings diminishes their effectiveness.
- Some view Microsoft's warnings as a "cover your ass" maneuver due to the industry's inability to fully prevent LLM vulnerabilities.
- Similar criticisms apply to AI integrations from other major tech companies.