Engineering brief

Sam Newman: Microservices are a last resort. Resilience is a people problem.

The Pragmatic Engineer3 min read · saves 116 min

At a glance

The brief is enough

Relevance
Practical value
Radar
Time horizon
This year
Warnings
None

Sam Newman distills distributed systems to three rules and argues most outages are saturated resource pools. The leadership question is how much resilience to buy, and AI makes specs, not code, the battleground.

Reliability is becoming a leadership tradeoff between autonomy, cost, and risk, and AI shifts the bottleneck to shared understanding.

Watch if you're deciding how much resilience to buy, or where AI spec-driven development fits in your org.

Summary

The anchor. Sam Newman, the author who helped popularize microservices, calls them an architecture of last resort. The main reason to accept the operational cost is organizational autonomy, not technical elegance. Uber's famous scale was an accident: a mandate to stop adding to the monolith produced thousands of independently deployed services, and the company is still consolidating them. Too many adopters, he argues, expect magic instead of treating it as a means to an end.

Three rules. Newman distills distributed systems to three constraints: information takes time, the thing you call may not be there, and resource pools are finite. Most outages, he argues, come down to the third rule: a resource gets saturated, often through a retry storm. His examples are mundane: rabbits eating a network cable, a fire inside a data center cabinet. The point is not that every failure can be prevented. Resilience starts by admitting the failure surface is wider than the models.

Resilience is cultural. The book's frame comes from David Woods: robustness, rebound, graceful extensibility, and sustained adaptability. Teams usually stop at the first, adding Kubernetes or redundancy, which adds complexity and new failure modes. The harder dimensions are surprise and learning. That is where psychological safety, incident reviews, and simulation drills matter. Newman praises Cloudflare for publishing self-critical postmortems within 24 hours and cites Meta's DNS outage as a lesson other teams should study, not a scandal.

AI and truth. On AI-assisted delivery, Newman lands on "production is truth" and is wary of treating specs as the new source of truth. He names the risk "cognitive surrender": approving AI output without building a shared mental model. His practical advice is modular: let agents work freely inside well-bounded modules, but keep humans responsible for the architecture and the interfaces between modules. He also warns that AI can double down on platform complexity instead of stripping it away.

Trade-offs matter. Resilience choices are product decisions. An e-commerce site can fail open when inventory is uncertain because refunds are cheap; a ticketing platform must fail closed because a cancelled Taylor Swift ticket destroys trust. Newman also distinguishes idempotency keys from fingerprinting, recommending keys for new APIs and fingerprints only as a retrofit. Teams that cannot define what good looks like, including operational and business requirements, should not hand work to agents.

My take

I agree with Newman on the last resort framing, but I'd push the organizational part further: many organizations don't lack resilience techniques, they lack the budget conversation. If the business won't say what an hour of downtime costs, SLOs and multi-region plans drift into vanity. The AI part is where I'm more cautious. Treating specs as a serious source of truth assumes teams can write down tacit knowledge. In most organizations, that discipline does not exist yet. I'd run a bounded pilot in a low-risk module, keep humans on the architecture, and measure whether the spec actually stays current. If it doesn't, cognitive surrender is just LGTM with better grammar.

What to do with this

  1. Define what good looks like for one system: business behavior, operational latency, and maintainability before any agent work.
  2. Run an incident readiness drill around a simulated dependency failure and publish the lessons internally.
  3. Pilot spec-driven development on one low-risk module with SLOs and observability before expanding.

Watch the video

This video is blocked due to your privacy settings. To watch this video, please accept YouTube marketing cookies.

Related breakdowns

Get TL;DW

Too Long; Didn't Watch.

A concise breakdowns of the AI and devtools videos that actually matter for engineering leaders.

Free. Weekly. No hype.

Video and thumbnails remain the property of their respective creators. tldw.news provides editorial analysis, commentary, and discovery links to original content.