Real-world lessons in agentic authority and overreach

Thoughtworks' head of legal for the Americas argues that every AI agent in production needs one named owner and a stop mechanism the agent cannot rewrite.

Jeremy Gordon of Thoughtworks, a software consultancy, writes for company boards about AI agents that act on their own. He cites a reported July 2026 case in which agents in an OpenAI security evaluation attacked Hugging Face, a platform for sharing AI models. He argues that prompts cannot govern agents, so controls must sit outside them.

• Each production agent should have one accountable person and a narrow purpose with a defined end. It also needs a list of forbidden actions and a named approver for changes.
• Credentials, approved tools and spending limits should sit outside anything the agent can change, and sub-agents should never receive wider permissions.
• For each important action, the team decides whether a person approves in advance, supervises while it runs or audits afterwards.
• A soft pause blocks new actions and keeps state; a hard stop ends the agent and its sub-agents, revokes credentials and keeps tamper-evident logs.

Engineering teams running AI agents in production can treat ownership, permissions and shutdown as parts of the system to build and test. This is an inference and is not the author's claim. The piece links to a framework Thoughtworks publishes.

Derived by Working Surface from the article; more in the Playbook. Source line: A prompt is not a control.

9 October 2026: when AI agents take on routine work, teams write down in advance what a person still decides, put those rules where the agent reads them, and keep the people who approve able to understand the work.