Wooden crates slide down a long ramp toward a loading dock, and a fine net strung across the ramp catches one cracked crate while a figure in silhouette watches from a high platform, the whole scene drawn with wide empty margins on every side.

Engineering at Meta

Catching Bugs Before They Land: JiTTesting’s Human-on-the-Loop Approach at Full Scale

Sreeja Thummala, Sophie Zeng, Mark Harman, Weini Liu and Eric Liu · 29 October 2026

Company story · critique & review · agents · Human approval

Takeaway

Meta's JiTTests use AI models to write tests that catch bugs in risky code changes before they are merged, and contact engineers only about the strongest findings.

Summary

Engineers at Meta describe Catching JiTTests, a system that uses AI models to write tests aimed at bugs in a code change before it is merged. Meta's codebase, hundreds of millions of lines long, changes too quickly for ordinary testing to keep up. The system is now part of Meta's infrastructure, and engineers set its criteria and review its results in aggregate instead of checking every output.

Key points

Each night the system selects the previous day's riskiest code changes and writes tests designed to fail if a change introduced a bug.
A group of AI models and a separate set of fixed rules sort likely false alarms from real bugs and rank what remains.
Engineers are contacted only about highly ranked findings, rarely read the test code and mostly confirm whether a change in behaviour was intended.
Of the engineers contacted, 589, or 64%, engaged with the findings and judged 60% to be real bugs; 74 serious privacy bugs were caught in the first half of 2026.

Implication

An engineering team can decide in advance when an automated testing system may interrupt a person, and review its results in aggregate. This is Meta's own account of its own system.

Plays

Engineering

Send engineers only the highest-ranked bug reports from AI-written tests, after AI models and fixed rules remove false alarms, and ask whether each change in behaviour was intended.

Engineering

Each night, have AI models write tests for the previous day's riskiest code changes, chosen by a risk score, so that a test fails if a change introduced a bug.

Derived by Working Surface from the article; more in the Playbook. Source line: Every bug caught before it reaches production is one that end users never need to deal with.

Source issue

1 November 2026: Teams putting AI agents to work decide in advance which steps a person must still take, and that person carries more review and a greater duty to push back.