Claude Code

How the Claude Code team checks the code Claude writes

3 min read AI-generated

A 39-item checklist built from past outages, and Claude only checks the items that match the diff. After three failed rounds, it calls in a human.

Featured image for "How the Claude Code team checks the code Claude writes"

On the Claude Code team, Claude writes and reviews about 90 percent of the code. In a November 16 post on claude.dev, Addy Osmani describes how the team makes sure nothing breaks along the way. The gist: without verification loops you can run one or two Claudes at a time, because you have to check everything yourself. With them, you can run dozens.

Three kinds of tests worth the effort

Osmani’s first pick is end-to-end tests. They follow real user flows, like adding an item to a cart, checking out and landing on a confirmation page, and they’re hard to fool: the feature either works or it doesn’t. Unit tests are faster, but an agent can pass them and still break the feature.

Property tests check statements that must always hold, against thousands of generated inputs. Examples: encode something and decode it, and you get the original back; an account balance never goes negative. Because the inputs don’t come from the agent that wrote the code, they catch edge cases it missed. Differential tests pay off in migrations: old and new systems get the same inputs, and every difference is either a bug or an approved change.

How the team works

Every repo gets a skill called verify that boots the app and drives it, through the Chrome extension, a simulator or the service running locally. Claude uses it before every commit. If a check still fails after three rounds, Claude stops and brings in a human.

On top of that there’s a reliability checklist with 39 items, each with a trigger you can read from a diff and the incidents behind it. Claude only checks the items whose trigger matches, usually 3 to 10. Code Review reads its rules from the base branch, never from the pull request, so a change can’t modify the check that judges it. Every commit carries a stamp saying how it was verified, including “none.” A commit gate can block commits that never ran /code-review or /verify, and it starts out in observe mode.

A second example from the same house

A November 17 post by Vivek Nair shows how far this goes. Before Claude Docs launched, he had a Claude Code session load-test a staging environment for two weeks, find failures and ship fixes, with up to 30 subagents at once. Anything hard to undo, like schema changes, Claude had to bring to him. The result, per Nair: 4.6 million docs in the first six days and 60 server errors across 2 billion requests.

More agents need tougher checks

Both posts make the same point: the more you hand to Claude, the more work goes into checking it. You can build a small version of Osmani’s setup yourself. A verify skill and a hook that runs the tests before a pull request take an afternoon. The checklist from your own outages grows over time.

Sources

Claude CodeTestingCode ReviewAgentsAnthropic