Is the software factory working?

Cate Huston, part-time technology chief at Twill, measured six months of agent-written pull requests and found that automated safety checks cut emergency fixes but slowed merges.

Twill connects employers with senior candidates through referrals, and its two engineers let AI agents write most of the code. Cate Huston, its part-time chief technology officer, wrote scripts to sort the company's pull requests by type and to track reliability and speed. The results show the team spending more of its output on automated checks, with fewer same-day fixes and slower merges.

• From April to September, pull requests rose 24-fold, features fell from 66 to 16 percent of them, and automated safety checks rose from 12 to 36 percent.
• Same-day emergency fixes fell from about ten in August to one in September. In September, 89 percent of feature pull requests carried their own tests, up from 62 percent in April.
• A rule that every change needs tests sat in the agent's instruction file from the start, but it held only once test coverage became a required check. The agent is now also told to run mutation testing, which checks whether tests catch deliberate errors.
• The median time a pull request stayed open rose from 1.2 hours in August to 3.5 hours in September, partly because database changes get more human review.

Huston argues that a team improving its product with AI should improve its automated checks just as often, rather than waiting for incident reviews. Her figures come from a two-person company, and she says the set-up would need rethinking with one or two more engineers.

Derived by Working Surface from the article; more in the Playbook. Source line: If you’re iterating on your product with AI, you should be iterating on your guardrails too.

7 October 2026: once AI agents do most of the making, checking and choosing become the scarce steps, and teams protect them with author responsibility, automated checks and tools that point to risk.