Linkerd team propose service mesh-free way to run multi-cluster Kubernetes

The team behind service mesh Linkerd has aired a proposal for a simplified way of running a multi-cluster Kubernetes architecture without the problems associated with cross-cluster calls – and it doesn’t involve a service mesh.

Once people started using Kubernetes clusters, the next step, inevitably, was multiple clusters, which increases flexibility, but adds complexity.

Linkerd maintainer Thomas Rampelberg said in a blogpost it is possible to build multicluster architectures while addressing problems of observability, security and routing, by running its service mesh across multiple clusters and “aggregating metrics to an external Prometheus or Thanos, sharing service information in DNS, and using cert-manager to rotate certs on cluster Ingress controllers.“

But this has downsides around cross cluster calls, he continued, as they “don’t have full metrics, do not preserve source identity across the cluster boundary, and cannot be the target of traffic splitting”.

Most crucially, this approach ran counter to Linkerd’s own goal of not requiring applications changes, because the application had to distinguish between on and cross cluster calls.

The solution, he continued, was an approach dubbed service mirroring, which will “allow the same guarantees that a service mesh like Linkerd provides for in-cluster calls—identity, traffic shifting, etc—to also be applied to cross-cluster calls.

“With service mirroring in place, the full observability, security, and routing features of Linkerd apply uniformly to both in-cluster and cluster-calls, and the application does not need to distinguish between those situations.”

The service mirror is a Kubernetes Operator, which “mirrors a remote cluster’s services locally to provide service discovery and allow pods to refer to the remote services. It also manages configuring endpoints so that traffic goes to the correct IP address”.

Rampelberg noted that “The astute reader may note that nothing in service mirroring actually requires a service mesh. Instead, the service mirror is a standalone component that can compose well with other projects.

“This applies not just to the potential users of service mirroring, but also to the service mirror itself. For example, the service mirror’s gateway component is plugable by design, and can be implemented by projects such as Ambassador, Gloo and Traefik.”

Rampelberg solicited feedback on the idea, and said “We’re actively prototyping this solution today, and hope to have a functioning implementation very soon.”