Form3's Triple Cloud: Resilience & Risk Management

Beyond Availability Zones: Multi-Cloud Mastery

Form3's presentation on their triple active-active multi-cloud architecture is a masterclass in tackling complex distributed systems challenges for critical financial infrastructure. The key insight is the strategic shift from relying on cloud provider SLAs to building an in-house, cloud-agnostic platform. Their adoption of Kubernetes across multiple clouds, coupled with cloud-agnostic technologies like CockroachDB and NATS JetStream, is a powerful testament to achieving true resilience. The innovative DNS-based cross-cloud pod addressing and the custom 'cross-cluster pod disruption budget' (X-PDB) are particularly noteworthy engineering feats that demonstrate a deep understanding of Kubernetes primitives and a willingness to extend them to meet extreme availability requirements. This approach directly addresses regulatory concerns about cloud concentration risk, a vital consideration for any financial institution. The choice of Go for microservices also highlights a pragmatic decision for performance and maintainability in a distributed environment. The presentation effectively articulates the 'why' behind their architectural evolution, driven by customer needs and regulatory pressures, making it highly relevant for organizations operating in highly regulated and availability-sensitive sectors.

However, while the presentation showcases impressive technical achievements, it also implicitly highlights the significant operational overhead and expertise required to maintain such a complex system. The decision to run independent Kubernetes clusters in each cloud, while enhancing fault tolerance, means managing three control planes and their associated complexities. The reliance on custom solutions like X-PDB, while effective, introduces bespoke code that needs ongoing maintenance and could potentially become a point of failure if not rigorously managed. The presentation doesn't delve deeply into the cost implications of running three active-active cloud environments, which is likely substantial. Furthermore, the 'when not to' aspect of the title is only lightly touched upon, leaving room for a more thorough discussion on the trade-offs, the suitability for smaller organizations, or use cases where the complexity might outweigh the benefits. The success of this architecture is heavily dependent on a highly skilled distributed systems engineering team, which is a significant barrier to entry for many. Despite these considerations, the Form3 model offers a compelling blueprint for achieving unparalleled resilience, particularly for financial services and other mission-critical applications.

Key Points

• Form3 evolved from a single-cloud AWS architecture to a triple active-active multi-cloud setup to meet regulatory demands and customer needs for cloud exitability.
• Key architectural goals included active-active-active resilience across clouds, adopting cloud-agnostic technologies, and avoiding cloud-specific services to simplify development and maintenance.
• The v2 architecture utilizes Kubernetes clusters in AWS, Google Cloud, and Azure, with a global load balancer and client-side load balancing for requests.
• Core technologies enabling this multi-cloud setup are Go for lightweight microservices, NATS JetStream for a unified, multi-cloud message broker, and CockroachDB for a distributed, PostgreSQL-compatible database.
• Significant engineering challenges were overcome, including cross-cloud Kubernetes pod discovery and addressing (via custom DNS forwarding) and ensuring high availability for distributed databases (via a custom cross-cluster pod disruption budget - X-PDB).

---

📖 Source: Presentation: How To Run on Three Clouds at Once, and When Not To