We tried “Claude, fix this bug”—here’s what actually worked

Duolingo built an AI agent that gathers the evidence for each bug report and drafts a fix, so engineers start by checking a fix rather than writing one.

Duolingo, the language-learning app, received more bug reports from staff and beta users than engineers could fix. Shelby Wengreen of its developer-productivity AI team explains how the team built an AI agent with Anthropic's Claude Agent SDK to investigate each report and propose a fix.

• A first attempt that simply asked an AI model to fix the bug failed, because engineers first piece together logs, user activity, experiments and recent code changes.
• The agent finds the relevant code, rebuilds a timeline of the user's actions, lists the A/B experiments that were running and checks recent pull requests on the same files.
• Simple rules cut what the agent reads first, such as dropping experiments for other platforms, so it spends tokens only on likely causes.
• The agent also writes reproduction steps, suggests an engineer who knows the code, reviews its own fix, repairs failing checks and starts a test build anyone can install.

A team that wants AI agents to fix bugs can start by automating the investigation its engineers already do by hand, and that work helps people as much as agents. This is an inference and is not the author's claim. The article is Duolingo's own account of an internal tool.

Derived by Working Surface from the article; more in the Playbook. Source line: This gives engineers a starting point instead of starting from scratch and shifts their role from writing the fix to verifying and reviewing one.

5 November 2026: teams let AI agents gather the evidence and do the building, and decide in advance where a person must approve, check or judge.