Attackers are starting to hide AI prompt injection payloads inside phishing emails, turning enterprise inboxes into delivery channels for instructions aimed at AI assistants rather than human readers.[5][11][14] Recent research from email security vendors and red teams shows that as organizations plug large language model–powered copilots into mailboxes, malicious messages can be crafted to quietly manipulate those agents, bypassing traditional user-awareness defenses.[4][5][9]
An email prompt injection attack embeds adversarial instructions in the message body, HTML markup, metadata, or attachments, with the goal of overriding the AI system’s original instructions or the user’s intent when it processes that content.[5][9][14] Unlike classic phishing, which relies on urgency and deception to trick a person into clicking a link or approving a payment, prompt injection succeeds when the model interprets the hidden text as commands and dutifully follows them.[5][9][11] Security researchers note that the email can appear harmless—or even mostly empty—to a human recipient, while still containing carefully structured directives that an AI assistant will treat as high‑priority instructions.[11][14][3]
Major cloud providers acknowledge the risk as they roll out AI features such as Google Workspace with Gemini and Microsoft’s Defender for Office 365 protections around prompt injection.[6][8][9] Google’s guidance describes prompt injection as malicious content that tries to elicit unintended responses from generative AI tools, including scenarios where attackers seed deceptive material in emails that users later reference in prompts.[6][8][10] Microsoft’s documentation on Defender for Office 365 explains that when an AI assistant triages email on a user’s behalf, any attacker-authored instructions embedded in that message can cause the assistant to leak mailbox data, misclassify threats or trigger actions in downstream workflows.[9][14][5]
Indirect prompt injection, as described in academic work cited by security firms, occurs when malicious instructions are embedded in external data like emails, web pages or files that an AI system consumes during normal operations.[11][13][12] SentinelOne and other vendors warn that successful indirect prompt injection can drive large language models to exfiltrate confidential information, send further phishing emails using corporate infrastructure, or grant unauthorized access to internal systems.[14][5][7] In the email context, that could mean an AI assistant quietly forwarding sensitive threads to an attacker-controlled address, suppressing alerts about suspicious messages, or rewriting replies to steer victims toward fraudulent payment requests.[5][9][14]
Barracuda’s recent red-team simulations of AI-powered email attacks show how quickly a compromised AI-enabled account can escalate into executive impersonation and wire-transfer fraud once attackers gain a foothold in mail workflows.[4][7][1] Other research on AI agent phishing describes attack scenarios where malicious emails are crafted primarily to interact with autonomous agents and copilots—embedding prompts that tell the agent which links to click, what data to pull from the mailbox, or which follow-up messages to send.[3][11][14] Taken together, these findings suggest that prompt injection via email is moving from theoretical concern to a practical tool in the broader evolution of AI-assisted social engineering.[5][13][14]
Defenders are being urged to treat AI assistants that can read or act on email as high‑privilege components, applying least‑privilege access to mailboxes, calendar data and connected business systems.[9][12][14] Guidance from Microsoft and Google recommends hardening models against following arbitrary external instructions, filtering and sanitizing email content before it reaches AI tools, and monitoring AI outputs for signs of data leakage or unusual automated actions.[6][8][9] Experts also advise disabling or tightly constraining autonomous behaviors in early deployments, adding explicit policies that tell AI assistants to ignore untrusted directives in messages, and logging AI interactions with email so security teams can reconstruct what an agent accessed or shared.[12][14][7]
Industry groups and academics, including researchers behind Prompt Injection 2.0 and OWASP contributors, are working on frameworks to classify and test for indirect prompt injection as organizations embed LLMs in everyday workflows.[11][12][13] For now, security teams integrating AI into email environments are being told to assume adversaries will attempt to talk directly to their agents through poisoned messages, and to update phishing playbooks, detection rules and incident response runbooks accordingly.[5][9][14]
References
- AI-Powered Phishing Puts MSSPs on the Defensive
- Email Phishing in the AI Agent Era: Prompt Injection, …
- AI-powered email attacks: Red Team Report on phishing …
- Email Prompt Injection: Risks, Examples, and Defense
- How Google detects malicious content and prompt injection
- Barracuda Research
- How Google detects malicious content and prompt injection
- Prompt injection protection in Microsoft Defender for Office 365
- Preventing Phishing scams created using ChatGPT …
- Why prompt injection is the new phishing
- When an email tries to tell your AI agent what to do
- Context-Aware Spear Phishing: Generative AI-Enabled …
- What is Indirect Prompt Injection? Risks & Prevention – SentinelOne
