Your coding agent shouldn't grade its own homework
Jeff An · 13 November 2026
Read the originalMomentic stopped letting coding AI agents test their own work after finding most of their unit tests useless, and gave the job to a separate testing agent.
Jeff An is chief technology officer of Momentic, a company that sells AI software testing, and leads an engineering team of eight that uses background AI agents. His engineers found that agents often declared features complete when basic functions failed on first use. Momentic now separates the AI agent that writes code from the one that tests it.
- A human-reviewed audit of 1,000 AI-written unit tests in Momentic's codebase found only 9.6% were useful checks; 22.8% duplicated existing tests and 22.4% relied too heavily on mocks.
- An argues that tests an agent writes for its own code share the assumptions behind that code, so they miss behaviour nobody specified.
- The testing agent receives only the original requirements and the running app. It starts with none of the coding agent's history, and the coding agent cannot change its acceptance criteria.
- Momentic tests with AI models from five providers, because research finds AI models can favour their own outputs. In its own benchmark, its testing agent found three to five times as many bugs.
An engineering team can keep the agent that checks work apart from the agent that did it, and test live behaviour rather than count passing unit tests. This is a vendor's account of an approach its product sells, and the benchmark is its own.
Audit a sample of AI-written unit tests by hand and count how many would catch a real defect.
Give a separate testing AI agent only the original requirements and the running app, so it checks behaviour without sharing the coding agent's assumptions.
Derived by Working Surface from the article; more in the Playbook. Source line: It's that they often test what they built, rather than what they should have built.
14 November 2026: as AI agents produce more code and tests than people can inspect, teams are writing down which work gets checked closely, by whom or by what, at what cost, and how they will know it worked.