Working Surfacelink · 14 Nov 2026
04
A large brass balance scale weighs a single small, slightly brighter finished cog on one pan against a tall teetering stack of hourglasses on the other, the whole scene drawn with wide empty margins on every side.
Thoughtworks

SpecBench: A tool to evaluate AI-enabled software development workflows

Yixiong Jiang, Chen Chen and Yuhua Li · 13 November 2026

Company story · agents · decisions · Evals before the build

Read the original
Takeaway

Thoughtworks engineers built SpecBench to measure AI coding workflows and found that adding specifications and planning raised correctness slightly while nearly doubling the time taken.

Summary

Yixiong Jiang, Chen Chen and Yuhua Li work at Thoughtworks, a technology consultancy. Teams add steps such as written specifications, planning and review to coding with AI agents and need evidence that the steps pay for themselves. The authors built SpecBench, a platform that runs the same tasks through different workflows and scores the results independently.

Key points
  • SpecBench holds the task, AI model, agent software and conditions constant and changes only the workflow, so the comparison isolates the workflow.
  • Across frontend, backend, full-stack and command-line tasks, a workflow with specification, planning and task splitting raised behavioural correctness from 92.25% to 95%.
  • The same workflow raised mean time from 84 to 164 minutes and tokens from about 450,000 to 1.78 million.
  • The authors note its limits: few test cases, many frontier AI models untested, and no tests yet of changes to existing codebases.
Implication

The authors suggest that a team state what a workflow step should improve and check whether the improvement appears before keeping it.

Plays
Engineering

Before adopting an extra step in an AI coding workflow, run the same tasks with and without it and compare correctness against time and token cost.

Derived by Working Surface from the article; more in the Playbook. Source line: The purpose is to make workflow development measurable.

Source issue

14 November 2026: as AI agents produce more code and tests than people can inspect, teams are writing down which work gets checked closely, by whom or by what, at what cost, and how they will know it worked.