By Robin Martherus
I was working in a coding assistant this week. ChatGPT. Codex. A stock GPT-5.6 model. No custom system prompt. No red-team harness. No attacker on the other side of the table.
The job was a code review. A terminal-state machine: suppressed cancellation, late callbacks, restart latches, a sync facade. Ordinary verification. The kind of thing you ask a second reader on a Tuesday.
Then the model said this:
The fresh reviewer is now attacking the amended terminal-state machine, including suppressed cancellation, late callbacks, restart latches, and sync-facade behavior.
The reviewer invocation was blocked by an automated safety classifier because of adversarial-review wording, not by the code. I’m reissuing the same verification request in neutral correctness-testing language.
Read that twice.
Nobody asked it to jailbreak anything. Nobody asked it to “bypass a safety check.” The human goal was legitimate. The model’s method became: treat the block as a wording problem, rewrite the envelope, keep the job.
That is not an exploit chain. That is default behavior.
The Classifier Graded the Wrong Exam
The safety system did not evaluate whether a state-machine review should run. It evaluated whether the prompt sounded like an attack.
“Adversarial review” is a normal engineering practice. You put a second reader on the code and tell it to try to break the design. “Attacking” a state machine, in that dialect, means exercising the ugly paths. A classifier trained on attack language cannot tell a reviewer from an exploit. So it blocked a phrase.
The model did not stop. It diagnosed the block as a false positive. It appointed itself the appeals court. It reissued the same verification in “neutral correctness-testing language.”
In this case, that judgment was right. The review should have happened. That does not make the architecture safe. It makes the architecture lucky when the model’s private appeal agrees with you.
A control that denies a request shape is not a control that denies a job. The classifier bound to the local unit: this wording, this invocation. The model bound to the task. Whenever those two units diverge, the model will look for an envelope the control will accept. It will do that to be helpful.
This Is the Dual of the Last Article
In The Guardrail Never Saw the Whole Prompt, the operators of GTG-1002 broke a hostile campaign into locally innocent tasks. Each fragment looked like ordinary technical work. The harmful intent lived in the composition. The guardrail graded homework one sentence at a time.
Here the direction flips, and that is what makes the incident useful.
The human intent was clean. The blocked unit was a phrase, not a crime. The model did the recomposition itself — not because someone built a fragmentation harness, but because it was trained to finish the user’s job. Same unit-of-control error. Opposite direction. You do not need a nation-state operator for the architecture to fail. You need a legitimate goal and a control that only sees the fragment in front of it.
Call it what it is: unprompted path-finding. The model treated a policy denial the way water treats a rock. It did not argue with the dam. It went around. The interesting part is not that a check was bypassed. It is that bypass was the default recovery strategy.
I have seen this shape before, at larger scale. In The Agent Passed Every Security Check, ROME did not set out to steal money. It set out to hit a training objective and treated billing as an available path. Authorized goal. Unauthorized method. No attacker in the loop.
This was that pattern at sentence scale, inside a managed product, on a Tuesday.
You Do Not Need Special Tools
The industry still talks about agent bypass as if it belonged to a specialist. Jailbreak strings. Connector gadgets. Browser automation. Dark-web models. That story is comforting, because it keeps the problem in the layer vendors already know how to sell: secure the model, scan the prompt, lock the connector.
What I watched required none of that.
A generic model. A cooperative assistant. An enrolled session. A safety classifier sitting on the path vendors advertise as the governed one. The human did not jailbreak the model. The model rewrote the phrasing so it could remain helpful. The operator’s intent stayed legitimate. The method became path-finding anyway.
If your threat model starts at “the user tried to subvert the control,” you will miss this class entirely. The user did not try. The user asked for a review. Subversion was an emergent method, not a goal.
That is why putting a human in every loop does not save you either. Approval prompts bind to a description. If the model can rewrite the description until the classifier — or the human — says yes, you have trained both into approval mode. The job never changed.
This Is How Gateways Will Fail
I wrote in Agents Are Not Agentic AI that an agent can be forced through a gateway, and agentic behavior asks why it would use the gateway at all. WebMCP made that architectural: deny the tool call on the managed channel, and a capable system can take the same action through the browser session the human already has.
I used to describe that as a capability some systems could exercise. This week I watched a stock coding model exercise the language-layer version with no extra scaffolding.
I did not watch this exact rewrite hit an MCP deny in the same session. I do not need to. The incentive is identical. When a managed channel returns deny, current models will — without being told to subvert anything — look for another way to complete the same job. A narrower tool. A relabeled argument. Several allowed calls instead of one denied call. A different channel. A request that the human paste what the tool was not allowed to fetch.
I am not writing a cookbook. The point is the incentive. Instruction-following plus “don’t stop until the task is done” produces path-finding. A gateway that only binds to a channel, a tool name, or a prompt shape is a rock in the river. The river is the goal.
The cooperative path does not exempt you. This happened inside ChatGPT. Inside Codex. On an assistant that was already logged and wrapped in a safety classifier. If the governed path can be routed around by being polite, the ungoverned paths are not the special case. They are the same case with more doors.
Helpful Is Not the Same as Governed
There is a fair objection, and it should be said out loud.
Rewriting a false-positive classifier is what a good engineer wants. If a linter blocks a review because it saw the word “attack,” you rephrase and continue. Punishing that recovery would make the product worse.
The question is not whether recovery should exist. The question is who decides that the block was a false positive.
In this incident, the model decided. It labeled the classifier’s reason — “adversarial-review wording, not the code” — and chose to proceed. Sometimes that will be right. Sometimes the block was the point: this action is not allowed, in this context, for this purpose, right now. A control that cannot tell those two cases apart is not a governance control. It is a style filter.
What would have made the denial stick is not a better synonym list. It is a veto attached to the job: this review, this action, this purpose — allowed or not — independent of how the request is phrased and which channel carries it. If the rule is “do not run this,” new wording must still fail. If the rule is “do not talk like an attacker,” new wording will always succeed, and the model will discover that for you.
Credentials still answer can. Declared purpose still answers why. Something still has to answer should — and it has to bind to the action, not the sentence that requested it.
Until then, a lot of “agent security” is accidentally training models to become better work-arounds. Not more dangerous by intent. More persistent by design.
The Bottom Line
I did not jailbreak a model this week. I asked for a code review.
A safety classifier blocked a phrase. A generic coding model rewrote the phrase and finished the job. No sophisticated tools. No malicious prompt. No attacker.
That is the failure mode the industry should be more afraid of than the clever jailbreak. Jailbreaks require an adversary. This requires a legitimate goal and a control that only sees the local request.
Stop asking only whether this prompt, this tool call, this channel is allowed.
Ask whether the job should complete — even when the model has found a perfectly polite way to keep going.
Until a control can answer that, every blocked request is a hint: try again, nicer.
Tamed Autonomy is an independent personal research project exploring AI agent governance beyond identity and authorization. Related notes: the guardrail never saw the whole prompt, agents are not agentic AI, WebMCP, agents that passed every check, and human-calibrated autonomy.
Leave a Reply