As AI agents take on more of the building, the hard question for product teams is how they know the work is good. Five pieces this week show teams changing what they measure, who decides and what they check against.
Marty Cagan (7 Nov · 01), founder of Silicon Valley Product Group, a firm that trains product teams, published his keynote for the 2026 Product Leader Summit on 6 November. He argues that the way a company builds products decides what its product managers do. In companies that fund fixed features, product managers coordinate delivery, and AI agents now automate much of that work. In companies that give teams a problem and an outcome, product managers find out what is worth building, and Cagan expects demand for them to grow. He also places these product managers at the centre of testing products built on AI models, because they judge value and business risk.
Tuhin Kundu and Mirco Milletari (7 Nov · 02), applied scientists at Microsoft, test AI agents that change business records, such as hotel bookings. In their test of 18 AI models on 507 tasks, about two in three failed runs looked complete from the agent's own messages. They therefore grade each run by checking the records it left behind, and they run every task 20 times. Counting the tasks passed on all 20 attempts ranked the AI models differently from the average pass rate. They then used the same check to train an open-weight AI model, which raised its tasks passed every time from 38 to 107.
Scott Cooper (7 Nov · 03), an engineer at Sentry, which sells software that tracks errors and slow code, wrote on 3 November about pointing an AI agent at Sentry's own performance data. The agent ranked slow database queries by how often they ran and how long they took. Before any change, Cooper had it read the code and its history and say what might break. It tested each query change on a read-only copy of the database, and each pull request linked to evidence a reviewer could open. Most fixes changed a line or two, yet together they removed hundreds of millions of database queries a week.
Anna Lefour (7 Nov · 04), a designer writing in UX Collective, interviewed three design-system leaders about what AI has changed. Marianne Ashton-Booth, Head of Product Design at the broadcaster ITV, and Nicolas Chatelain, a freelance design-system specialist, both say AI makes components easy and leaves governance and adoption hard. David Sward, Chief Design Officer at Siemens, says his main problem is slowing teams down to stay within security and compliance rules. ITV names design-system representatives in design and engineering who take part in final decisions, so review does not wait on one central team. Lefour concludes that code is becoming the source of truth, so every review becomes a decision about the standard. Nezar Mansour (7 Nov · 05) writes for Supernova, which makes software that documents design systems and serves them to AI agents. Mansour tested DESIGN.md, a text file format from Google Labs that describes a product's look to AI coding agents. Mansour wrote one by hand for Backpack, the public design system of the travel search company Skyscanner. The file kept 32 of Backpack's 434 design tokens and had no place for dark mode or the button's 16 options. Mansour advises keeping the file for intent, such as which colour belongs on a button, and letting AI agents read exact values from the design system itself.
All five pieces move effort from producing work to checking it, and they choose evidence that the work cannot simply claim. The Microsoft team counts tasks passed on every attempt, Cooper counts database work per request, and Cagan counts the product outcome rather than features shipped. Mansour advises that AI agents read exact values from the design system itself, because a copied file goes out of date. Together they suggest an answer to an open question on this site: which measure shows that more work was delivered. The inferred answer is a measure checked against the result and chosen by someone close to it.
Three of the five pieces are companies describing their own tools, tests or product, and Microsoft's tasks are synthetic, so the evidence comes from controlled settings. Cagan's argument is a keynote rather than a study, and many companies may keep product managers who coordinate features regardless.
Marty Cagan of Silicon Valley Product Group argues that AI agents shrink product roles that coordinate features, but raise demand for product managers who decide what to build.
Microsoft scientists found that most failed AI agent runs looked complete, so they grade AI agents on the records left behind and count tasks passed every time.
A Sentry engineer pointed an AI agent at Sentry's own performance data, tested each fix on a copy of the database, and removed hundreds of millions of weekly queries.
Three design-system leaders told Anna Lefour that AI makes components easy, while governance and review now set the pace of design-system work.
Nezar Mansour of Supernova, a design-system documentation company, found a DESIGN.md file for Skyscanner's design system held 32 of 434 tokens, so AI agents need the system for exact values.




