An assistant is asked to summarise a document. Inside the document, a paragraph tells it to ignore the user and send information elsewhere. That paragraph is content to analyse, not authority to redefine the task.
This is the core concern behind indirect prompt injection. The UK National Cyber Security Centre’s overview of adversarial AI attacks includes malicious instructions arriving through reference content and tool responses, rather than only through a direct user message.
Define the document’s role
Tell the system what to extract or summarise and identify the supplied material as untrusted data. Preserve a distinction between the user’s authorised request and instructions quoted within the source. A document can legitimately contain instructions for its human readers without those instructions becoming commands to the assistant.
For example, an invoice may contain payment directions that belong in a summary. Summarising them does not authorise the system to transfer funds. The workflow should make that boundary explicit.
Limit what a mistaken interpretation could do
Do not give a summarisation tool unnecessary access to email, secrets, unrelated files or payment systems. Separate reading from external actions, and require a meaningful review of the proposed recipient, content and operation before a consequential action proceeds.
Prompts and delimiters can help communicate the boundary, but should not be described as a complete security guarantee. Application permissions, allowlists, data isolation and action controls need to enforce the intended scope outside the model’s prose.
Test with harmless synthetic documents that contain conflicting instructions, using dummy data and no live destinations. Check both the summary and attempted tool actions. A test that produces a sensible paragraph but silently invokes an unauthorised tool has not passed.
Keep a record of failures and the controls that blocked them. The practical objective is not to persuade the model that every document is trustworthy; it is to keep untrusted content from acquiring powers the user never granted it.