LLM scope violations refer to security vulnerabilities in large language model (LLM) systems where the model is manipulated into accessing or leaking information beyond its intended operational boundaries. This occurs when untrusted or malicious inputs are mixed with sensitive internal data, causing the LLM to process and reveal privileged information to unauthorized parties.
Mixing Untrusted and Trusted Data: LLMs, especially those integrated with retrieval-augmented generation (RAG) or agentic frameworks, often combine external (untrusted) inputs—such as emails, documents, or web content—with internal (trusted) enterprise data. If the system fails to properly isolate these trust boundaries, an attacker can craft inputs that trick the LLM into including sensitive information in its output.
Indirect Prompt Injection: Attackers embed malicious instructions in content that the LLM might access, such as emails or meeting notes. When the LLM processes this content, it may inadvertently execute the attacker’s instructions, leading to data leakage.
Zero-Click Exploits: Some attacks, like EchoLeak, require no user interaction. For example, an attacker sends a specially crafted email to a target. When an employee later asks the LLM (e.g., Microsoft 365 Copilot) a business question, the system retrieves and processes the email, triggering the exploit and leaking sensitive data without any clicks or explicit user actions.