Anthropic published an engineering blog post on September 14, written by Sachin Malhotra, about an internal service that got overwhelmed and had to be rebuilt. The service is called test impact analysis, and its job is narrow: given an incoming code change, decide which tests must run and which can be skipped. It sounds like a backend utility, but the reason it buckled says a lot about what's happening more broadly.
The multipliers first
The first set of numbers in the post is about output: Anthropic engineers now merge, on average, eight times as much code per quarter as they did in the 2021-2025 period, and 80% of that code is written by Claude. Claude also carries a heavy share of reviewing and approving pull requests.
The second set is about pressure: the total number of tests in the codebase is up tenfold, and the number of CI jobs rose 25x within six months.
The third set says the most about the pace of the problem. The team shipped three rounds of stopgap mitigations to keep the system standing, and each one bought progressively less time: 70 days, then 29 days, then less than a single day. The third fix was overwhelmed on the day it shipped.
Where the old service broke down
The original architecture had a single-process listener that wrote every test result to storage in sequence. A single writer can't scale horizontally, so once pull requests arrived faster than the listener could process them, a queue started to build. The post describes the listener as one that "starts to increasingly fall behind the PR queue."
The lag compounded. The post gives an example: a twenty-minute delay meant tens of thousands of test-result updates hadn't yet landed in storage. Since test selection depends on historical results, stale history meant an unreliable test set. On top of that, the process also had a memory leak and hit its ceiling every day by the afternoon.
What it was rebuilt into
The rewrite moved state out of the process. The new architecture introduces an in-memory data store: any listener worker can handle any result, and each one only appends to a log without holding state itself, so workers can be added on demand. A separate consumer process folds the log into per-test history every few seconds. When the selector needs to make a decision, it queries that history directly.
The trade-off is higher cost. The post acknowledges this openly, in exchange for scalability and observability — which stage is slow, and where, can now be measured independently. One engineer completed the entire redesign in three weeks.
The 25x curve
25x in six months works out to roughly a 1.71x monthly compound growth rate (the sixth root of 25). Extrapolating that slope another six months forward gives 625x. The line that closes the post, "always plan for the exponential," isn't a rhetorical flourish — it maps onto a curve that has already been measured.
Extrapolating that curve is of course unreliable — growth rates can bend with internal policy, quota, or engineering-habit changes, and Anthropic hasn't disclosed absolute CI billing figures, what share of tests the new service filters out, or what latency looks like after the rewrite. How much money was saved can't be derived from what's public.
Still, the direction the pressure travels is clear. This April, GitHub stopped new sign-ups for three of Copilot's individual subscription tiers, citing compute consumption from long-running agent tasks that exceeded what the plans were designed for; earlier still, a surge in AI-generated commits pushed GitHub into an outage, and Microsoft had to borrow capacity from AWS to hold the line. The three episodes look different, but point to the same thing: code generation sped up first, then review, and every system after that sized around "the pace of a human writing code" gets hit in turn. Test impact analysis was just the one that raised the alarm early this round.
Sources: CocoLoop, Anthropic Engineering Blog; the code-volume multiplier, test and CI job growth figures, and mitigation timelines are all as reported in that post.