GitHub rebuilds Git storage as commits jump 5x in a year

Brian Celenza, a principal software engineer at GitHub, wrote on the company's engineering blog on October 6 that GitHub is rebuilding its Git storage layer. The reason is stated plainly: AI agents have pushed write traffic past the limits of the existing architecture. In internal benchmarks, the new architecture lifts write throughput by up to 35x.

The post lays out a year's worth of growth figures:

The single busiest repository received about one billion requests in August alone. The post describes how agents write to repos this way:

"An agent in a tight loop commits or checkpoints after nearly every action"

The old architecture hit a write ceiling

GitHub's existing repository storage system is called Spokes. By default, each repository is fully replicated across multiple file servers' local disks — five copies in total. When a push updates a reference, a majority-based three-phase commit protocol keeps CI, the website, and API clients looking at a consistent view of the repository.

In this design, replicas do two jobs at once: protect against data loss, and absorb read traffic. The cost is that every replica has to take part in every write, so adding replicas to handle more reads makes writes slower. That trade-off was fine in the era when human developers drove most activity and reads vastly outnumbered writes. Once agents pushed up the write rate, the replica count itself became the bottleneck for writes.

The new architecture splits into three pieces

The first piece is minimizing coordination: only reference updates still require agreement across replicas; object storage, validation, and security scanning now run in parallel.

The second piece is separating storage from compute. Persistence is handed off to Azure Blob Storage, while a fleet of lightweight worker processes handles reads. As the post puts it:

"Separating the two lets us scale each one independently"

The third piece pulls background maintenance out on its own: compaction and garbage collection run on separate worker processes instead of competing with live requests for resources. Branch protection, required reviews, audit logs, and observability controls all remain in place.

A rough tally of the write load

Working from a 30-day month, 7.38 billion commits work out to roughly 2,850 per second; 3.35 billion pushes work out to roughly 1,290 per second, up from about 270 a year earlier. If every push still required five replicas to participate, the old architecture would have had to handle something in the neighborhood of 6,500 replica writes per second (a rough estimate that doesn't account for retries or maintenance jobs).

A 35x throughput gain against a 4.9x annual growth rate in pushes leaves several years of headroom on paper — assuming agent traffic's growth curve doesn't keep steepening, something GitHub itself isn't willing to forecast.

What the post doesn't say

The post gives no timeline for how many repositories have already migrated to the new architecture or when the full cutover will happen, and it doesn't specify what scale of repository or what level of concurrency the 35x benchmark was run against. It also doesn't address what the change might mean for users, such as whether rate limits will be adjusted.

GitHub has had several extended outages this year, and the post doesn't address those incidents or say whether the new architecture would help prevent similar ones. Celenza closes by previewing that the next post in the series will go deeper into the future architecture and the backstory behind this rebuild.

Sources: GitHub Engineering Blog, CocoLoop; monthly commit, push, and Actions-run figures verified against GitHub's official data.