Cluster Reconciliation: Managing Resources Across Multiple Clusters

Vallery Lancey at KubeCon + CloudNativeCon North America 2020

Suppose your organization runs multiple Kubernetes clusters, as failure domains or distinct POPs. You have a service that you wish to run on some (or all) of the clusters. How will you run it on all (applicable) clusters? How will you roll updates gradually? How will you handle “snowflake” clusters that aren’t in the right state, or gracefully adding/removing clusters in the group? In this session, we will describe the challenges associated with reconciling Kubernetes objects from a source of truth to arbitrary clusters, and how to solve those challenges. Aside from challenges, we will discuss the tradeoffs that can be made, especially in domain-specific designs. Expect to walk away with a better understanding of the challenges involved in running multiple clusters, knowledge of existing tools and techniques, and an idea of how upstream developers approach the problem space.