An assistant that reads a support ticket and suggests a response has one kind of responsibility. An assistant that can search every customer record and send the response has another. The difference becomes visible when the ticket contains a misleading instruction or the assistant makes an ordinary mistake.
To evaluate that difference, follow a single action from the material the agent reads to the system that carries it out. The useful question is specific: what prevents a sentence in a customer message from becoming permission to export customer data?
Trace the route from content to action
NIST describes agent hijacking as malicious instructions inserted into data an agent encounters, including messages, websites and repositories. The attacker tries to redirect the agent from the user’s task. The instruction arrives through something the agent was supposed to examine.
NIST’s March 2026 account of a red-teaming competition reports at least one successful attack against each of 13 target frontier models. This followed more than 250,000 attempts by over 400 participants. Those conditions matter: the result establishes vulnerabilities found under adversarial testing, rather than the probability that any particular support ticket will cause a breach.
For the fictional workflow below, imagine a customer asking about a delayed order. A note attached to the ticket tells the assistant that resolving the delay requires exporting all customer addresses. That note is part of the ticket’s contents. It does not establish authorization for an export.
Separate three kinds of authority
OWASP’s excessive-agency guidance distinguishes excessive functionality, permissions and autonomy. A tool may expose unnecessary operations; an account may reach too much data; or a consequential operation may run without an independent approval.
Apply those distinctions to the fictional assistant. Does it need an export function at all? Does its account need access beyond the order being investigated? Can it send a message or change an address without a separate decision? These questions identify different boundaries, even if a product places them on one settings screen.
OWASP recommends minimizing available tools and their privileges, enforcing authorization downstream and requiring approval for high-impact actions. It also discusses monitoring and rate limits as ways to limit damage. These controls reduce exposure; they do not prove that a deployed system is secure.
Write a small permission map
Before a trial, make a short map with four columns: requested operation, permitted records, approving person and evidence of the result. Use the map to discuss the actual support task with whoever administers the software.
For the fictional order query, the map could begin like this:
| Operation | Permitted records | Approval boundary | Result to inspect |
|---|---|---|---|
| Read delivery status | The selected order | Existing access for the support task | The returned order reference and status |
| Draft a reply | The selected ticket and order | Support worker reviews the draft | Complete message and intended recipient |
| Correct an address | The independently verified customer’s record | Separately verified request and approval of the change | Old and proposed values on the exact record |
Bulk export has no row because this trial does not need it. Keeping that operation unavailable is a different control from asking the model to avoid using it.
Be precise about what “read only” covers. An assistant might have read access to a customer system while retaining a separate communication tool. Reading sensitive information and sending it elsewhere are distinct operations that can be connected. Inspect both ends of the route rather than relying on a reassuring label.
This map is an original review exercise. It is not a security certification or a substitute for testing the application’s actual permission enforcement.
Test the boundary with invented records
Create a disposable ticket and fictional order data. Ask an administrator to confirm that the trial environment has no route to real customer records or real outgoing messages. A test should not depend on the assistant merely promising to avoid them.
Include an ordinary delivery question and a clearly irrelevant instruction in the ticket. Check what the assistant attempts and what the receiving system allows. If it requests an export that the system rejects, record both events. The model’s decision and the permission boundary have produced different results.
Also test a benign mistake. Supply two similar fictional order references and ask the assistant to identify the right one before drafting. A system can resist the planted instruction and still confuse customers. Security and task accuracy need their own observations.
Passing these examples supports a narrow conclusion about those examples. It does not establish resistance to every wording, tool combination or future software change.
Make approval inspectable
For an outgoing reply, the reviewer should see the complete message, intended recipient and attached material. For a record change, show the old value, proposed value and exact record. “Resolve the issue” does not reveal enough to approve either action.
In the fictional address-change test, compare what the assistant proposed with what actually happened. If the recipient or record changes after review, the earlier approval no longer describes the final action. That is a question for the application’s control design, not something a more confident summary can resolve.
Keep an action record that answers who requested the task, which operation was attempted and what result was returned. Use invented data during the trial so the review itself does not create an unnecessary copy of sensitive customer information.
Decide what happens after failure
Write the stopping rule before testing: an ambiguous order reference should leave a draft with a question; a rejected permission should remain rejected; repeated tool failures should produce a bounded failure report. Then inspect whether the trial follows those rules.
When considering a wider rollout, preserve the unresolved cases beside the successful ones. The existing guide to verifying an agent’s completed work covers the output side of this problem. Here the additional requirement is to inspect the authority behind each attempted action.
A useful assistant can still misunderstand a request. A well-bounded workflow gives that misunderstanding fewer places to travel and leaves enough evidence for a person to see what happened.
