Evals before the build

Abtin Aghagolian, chief technology officer of Pikd, a London augmented-reality company, argues that tests written before an AI agent writes code are now how engineers control its output.

Maggie Appleton, a research engineer at GitHub, no longer reads the code AI agents write for her prototypes and writes a spec saying how the agent will check its work.

Two Anthropic engineers advise companies modernising old code with AI agents to agree, before the work starts, what each change must prove and how much human review it gets.

Two specialists in testing AI products argue that people should read real AI sessions and note the failures themselves before an agent looks for errors or anyone writes a metric.

Warp's chief executive explains how AI agents can grade a sample of coding agents' past work against written criteria, so that people review the failures rather than every change.

Engineer and educator Matt Pocock uses short instruction files that make AI coding agents question him closely about a plan, which leaves him more time for strategic work.

Teresa Torres spent three weeks and sixteen experiments fixing one customer complaint about an AI product, starting by measuring how often the error occurred.