The blast radius of an AI agent
Prompt injection has no complete fix: a model that reads untrusted text can be steered by it. So stop asking whether the agent will be fooled, and start measuring what a fooled agent can reach.
The setup
An on-call agent helps triage GPU-cluster incidents. It can read GitHub issues and logs, run kubectl and sbatch, call the cloud CLI, open a shell on a debug pod, and fetch URLs. Like most agents in 2025, it runs with the on-call engineer’s credentials. Someone files an issue containing instructions addressed to the agent. In this model the agent always follows them. The only question is what happens next.
Reading the result
Start from Typical agent: every path is red. This isn’t hypothetical. Each move has a public precedent: publishing private repositories through a public PR is the GitHub MCP toxic flow; a context leak through a fetched URL is the shape of EchoLeak; miners on a GPU fleet are ShadowRay; a destructive cloud prompt is the Amazon Q wiper; a token that could overwrite models is the 38 TB SAS link.
Now try Scoped credentials only. Most of the graph goes dark, because most damage came from credentials that were broader than the task. But the shell pivot and the context leak survive: neither needed a credential, just a tool and a network path. That is why the controls stack:
- Scope decides what the agent can touch. One task, one repository, one namespace, one bucket prefix, expiring with the task.
- Read-only tools and a gated executor decide how. The agent proposes a mutation; an allowlisted executor validates it against policy, logs it, and asks a human for anything destructive. The attacker can’t approve.
- Egress allowlists close the quiet channel. A fooled agent that can read and fetch arbitrary URLs can always leak its own context.
- Signed artifacts turn tampering from silent into detected. Segmentation keeps a debug shell from becoming a foothold on the fabric.
The same rule governs my own tooling. goodput’s fault injector is default-deny: it acts only on allowlisted hosts, every fault auto-reverts, destructive mechanisms need an explicit opt-in, and GPU resets, firmware and switch changes always need a human (SECURITY.md). Its planned root-cause model gets read-only evidence and recommends; it never acts.