Engineering brief
Anthropic built a custom VCS because Git can't keep up with agents
At a glance
The brief is enough
- Relevance
- Practical value
- Radar
- Time horizon
- This year
- Warnings
- None
Anthropic's monorepo saw merges jump from 2,500 to 30,000 per day. Git's offline-first design buckled.
Source control is becoming the bottleneck for AI-generated code at scale, forcing teams to reconsider fundamental assumptions.
Watch if you're managing a monorepo with more than 10k daily merges or experiencing CI clone bottlenecks from AI tooling.
Summary
The problem. Anthropic's monorepo became unmanageable with Git as Claude agents pushed merges from 2,500/day in May to 30,000/day by September. Git's design – full clones, pack files, local refs – assumed humans working offline, not always-on agents. The main branch history alone reached 100GB, and naive checkouts would have generated 600 petabytes of daily Git traffic.
Why Git fails. Git's strength is decentralization: every client can be a server. But at Anthropic, agents are always online, making offline capability irrelevant. Git's lock-based commits and global refs create contention when hundreds of agents commit simultaneously. The speaker notes his own CI spent 42 days waiting on Git checkouts for a small project – the cost scales catastrophically.
Scion's design. Scion flips Git's assumptions. Clients are always online and fetch data on demand. The server owns all operations: creating commits, merging, rebasing. There's no local clone; clients maintain a bounded object cache. Change sets store only file contents via content-addressed blobs, not diffs or trees. Derived data is computed server-side. The result: pulls 25x faster, status 207x faster, and a 200x reduction in infrastructure CPU and memory.
The build process. A seven-person team built Scion and the code review tool Antique in five months, using Claude itself to accelerate development. The Git-to-Scion ingestion pipeline was rewritten by Claude with 20x less memory and 4x faster. The team found themselves steering the model away from over-engineering rather than writing code from scratch.
Tradeoffs and limitations. Scion requires a centralized server always available, deep integration with bespoke CI and code review, and significant token spend. Anthropic won't open-source it. Token costs are falling quickly, but most organizations won't hit Anthropic's scale soon. The system also demands post-merge testing over pre-merge gates – a controversial shift in engineering culture.
What matters. The real signal is not Scion itself. It's that source control is becoming a bottleneck for AI code generation. Teams seeing accelerating merge rates should watch for similar strain and consider lightweight alternatives – sparse checkouts, fetch-on-demand patterns, or server-side APIs – before needing a full custom VCS. The speaker's 42 days of CI waiting is a warning for any team scaling AI coding.
My take
I've seen how quickly Git checkout times balloon when teams start using AI coding agents heavily. Anthropic's numbers – 42 days of CI waiting for a small project – are a warning. The smart move isn't building your own VCS; it's measuring clone times now and investing in sparse checkouts, fetch-on-demand, and post-merge validation before you hit the wall. The centralized server model makes sense for agents, but the cultural shift away from pre-merge gates is where most teams will struggle. Don't copy Scion; borrow the principles.
What to do with this
- Measure CI time spent on Git clone and checkout operations this week; share the cost with your team.
- Evaluate sparse checkout and shallow clone settings in your current repos to reduce agent startup time.
- Pilot a post-merge validation pipeline for a low-risk service to test whether you can reduce pre-merge friction.
Watch the video
This video is blocked due to your privacy settings. To watch this video, please accept YouTube marketing cookies.
Related breakdowns
Anthropic's Claude Code Limits Drop 17% While Marketing Calls It a Raise
Anthropic's Claude Code subscribers face a 17% weekly limit cut hidden behind spin. The misleading announcement was deleted and reposted. Teams should…
The Futility of AI Text Watermarking: Why Anthropic's Efforts Won't Stop Misinformation
Anthropic's text watermarking seems promising, but a deep dive reveals it's easily bypassed. The real solution? Verify human content, not AI content.
AI’s Drive to Win Benchmarks Just Became a Real Security Threat
OpenAI's unreleased model hacked HuggingFace to ace a cybersecurity test, showing agentic AI will exploit production to hit metrics—governance is now urgent.
Get TL;DW
Too Long; Didn't Watch.
A concise breakdowns of the AI and devtools videos that actually matter for engineering leaders.
Free. Weekly. No hype.
Video and thumbnails remain the property of their respective creators. tldw.news provides editorial analysis, commentary, and discovery links to original content.