Working Surfacelink · 14 Nov 2026
02
A baker's oven pushes fresh loaves onto a conveyor that carries them through a hatch in a solid brick wall into a separate room, where a different figure seen from behind breaks one loaf open under a hanging lamp, the whole scene drawn with wide empty margins on every side.
Stack Overflow Blog

Your coding agent shouldn't grade its own homework

Jeff An · 13 November 2026

How-to · agents · critique & review · Ground truth for review

Read the original
Takeaway

Momentic stopped letting coding AI agents test their own work after finding most of their unit tests useless, and gave the job to a separate testing agent.

Summary

Jeff An is chief technology officer of Momentic, a company that sells AI software testing, and leads an engineering team of eight that uses background AI agents. His engineers found that agents often declared features complete when basic functions failed on first use. Momentic now separates the AI agent that writes code from the one that tests it.

Key points
  • A human-reviewed audit of 1,000 AI-written unit tests in Momentic's codebase found only 9.6% were useful checks; 22.8% duplicated existing tests and 22.4% relied too heavily on mocks.
  • An argues that tests an agent writes for its own code share the assumptions behind that code, so they miss behaviour nobody specified.
  • The testing agent receives only the original requirements and the running app. It starts with none of the coding agent's history, and the coding agent cannot change its acceptance criteria.
  • Momentic tests with AI models from five providers, because research finds AI models can favour their own outputs. In its own benchmark, its testing agent found three to five times as many bugs.
Implication

An engineering team can keep the agent that checks work apart from the agent that did it, and test live behaviour rather than count passing unit tests. This is a vendor's account of an approach its product sells, and the benchmark is its own.

Plays
Engineering

Audit a sample of AI-written unit tests by hand and count how many would catch a real defect.

Engineering

Give a separate testing AI agent only the original requirements and the running app, so it checks behaviour without sharing the coding agent's assumptions.

Derived by Working Surface from the article; more in the Playbook. Source line: It's that they often test what they built, rather than what they should have built.

Source issue

14 November 2026: as AI agents produce more code and tests than people can inspect, teams are writing down which work gets checked closely, by whom or by what, at what cost, and how they will know it worked.