Engineering brief
Payments on Three Clouds, Then Back to Two: A Resilience Reality Check
At a glance
The brief is enough
- Relevance
- Practical value
- Radar
- Time horizon
- Now
- Warnings
- None
Form3 built triple-active multi-cloud for the UK regulator. In the US, latency and market expectations forced a switch to active-standby.
Multi-cloud decisions are shaped by regulations and market expectations, not just technical merit.
Watch if you're designing multi-cloud or disaster recovery for a regulated industry or a market with latency constraints.
Summary
The trigger. Form3, a UK payments company, was forced into multi-cloud when banking regulators worried about concentration risk. One large bank customer demanded the ability to exit any cloud provider. The company decided to build an active-active-active platform across AWS, GCP, and Azure, treating each cloud as an availability zone.
The architecture. They moved from Java to Go for lighter deployments, ran separate Kubernetes clusters per cloud, and used NATS and CockroachDB as single cross-cloud data stores. To connect pods across clusters, they built a custom DNS forwarding mechanism with static IPs and network load balancers. They also created two operators: XPDB, a cross-cluster pod disruption budget, and CLO, a cluster lifecycle operator to manage node pools at scale.
The proof. During a Google Cloud outage, payments kept flowing while pods crashed in GCP. Traffic shifted to the other clouds automatically. The speaker described opening his laptop on a train to see only low-level alerts, while all payments continued without interruption. This validated their approach of treating a cloud provider like a failure domain.
The market reality. When entering the US, customers expected geographic redundancy, not multi-cloud active-active. Latency across the continent made sync writes infeasible. Form3 settled on an active-standby design: AWS East with a warm standby on GCP West, using backups and non-zero RPO/RTO to meet customer expectations. The US market was comfortable with this trade-off, as it was the norm.
The failover decision. During an AWS incident, they considered failing over but decided to wait. AWS recovered, and they realized failing over would have caused a longer outage. This led them to invest in faster failover: logical replication in CockroachDB and NATS stream replication, reducing RTO and enabling customer-specific DR testing without disrupting others.
The trade-offs. Multi-cloud is expensive and complex, requiring a robust platform team. Form3 also learned to avoid over-granular infrastructure as code, which created maintenance overhead. The talk's core lesson: resilience architecture must match market expectations and regulatory drivers, not just technical ideals.
My take
I'm impressed by the engineering discipline, but I'm also skeptical of the triple-active approach for most organizations. The cost and complexity are enormous, and the US experience shows that a simpler active-standby with fast failover can be more practical. The key insight is that resilience architecture must be driven by the market and regulatory requirements, not by a desire to be cutting-edge. The real takeaway for leaders: know your latency budget and your customer's risk tolerance before committing to multi-cloud.
What to do with this
- Ask your team to map regulatory and market resilience requirements before designing multi-cloud.
- Define a failover decision tree based on outage duration and recovery time.
- Evaluate whether active-standby with warm data replication meets your SLAs cheaper than multi-cloud active-active.
Watch the video
This video is blocked due to your privacy settings. To watch this video, please accept YouTube marketing cookies.
Related breakdowns
Context Bloat Will Drain Your AI Budget—Fix It with Just-in-Time Tools
Unused MCP servers silently multiply token costs by 175x. Gateways and registries enable just-in-time tool delivery—but real-world proof is absent.
AI’s Real Bottleneck: Context, Not Compute
CAST’s graph halved token use and doubled accuracy, letting a team cut a mainframe modernization from 7.5 years to 3.
eBPF’s Pre-Hook Enforcement Changes Vulnerability Management
eBPF blocks exploits before they run—live CVE mitigation without reboots. Complexity is hidden behind tools like Tetragon/Cilium. Ready?
Get TL;DW
Too Long; Didn't Watch.
A concise breakdowns of the AI and devtools videos that actually matter for engineering leaders.
Free. Weekly. No hype.
Video and thumbnails remain the property of their respective creators. tldw.news provides editorial analysis, commentary, and discovery links to original content.