Cilium Cluster Mesh: Seamless Networking Across Cluster Boundaries

Operating highly available platforms for critical infrastructures (KRITIS) presents an architectural challenge: To achieve maximum fault tolerance, services are often deployed in multiple geographically separated data centers on independent Kubernetes clusters. However, in practice, these isolated worlds often need to communicate with each other—whether for querying metrics, accessing redundant databases, or coordinating workloads.

The solution to securely connect these clusters without building complex VPN constructs at the application level is Cilium Cluster Mesh.

1. What is a Cluster Mesh?

Cilium is a modern Cloud-Native CNI (Container Network Interface) based on the high-performance eBPF technology in the Linux kernel. With the “Cluster Mesh” feature, multiple Kubernetes clusters can be connected into a shared network infrastructure, while the control plane of each site remains strictly separate.

• Transparent Connectivity: Pods in one cluster can directly reach pods in another cluster via their IP addresses. The routing is abstracted at the network level, so the application does not need to be aware of the physical distance.
• Global Service Discovery: When a service is marked as a “Global Service,” Cilium recognizes it at all locations. If a local instance fails, traffic can be automatically and invisibly redirected to the healthy endpoint in another data center (Cross-Cluster Load Balancing).

2. Global Network Policies: Security Without Configuration Drift

In a KRITIS environment, “Default Allow” is not an option; every communication must be explicitly permitted. The problem with multi-site setups is often the manual synchronization of firewall rules: A rule active at site A is forgotten at site B, leading to errors in the event of a failover.

Cilium addresses this with identity-based security:

• Moving Away from IP Lists: Since IP addresses in Kubernetes constantly change, Cilium uses security identities. A rule then states: “The service frontend may only communicate with the service backend"—regardless of which cluster the respective instances are currently running in.
• Centralized Enforcement: By linking the clusters, security policies are consistently synchronized. A change to a global policy becomes active at all locations immediately. This significantly reduces the risk of human error during audits.

3. Transparent Encryption at the Node Level

Data exchange between data centers over public or shared lines must be encrypted. Cilium Cluster Mesh integrates this encryption (e.g., via Wireguard or IPsec) directly into the network layer.

• No Overhead for Developers: The application is unaware of the encryption. There is no need to manage certificates within the application (mTLS) because the network interface handles the protection of all traffic between nodes.
• Kernel Performance: Since processing occurs directly in the operating system kernel via eBPF, the performance loss compared to traditional user-space VPNs is minimal. This is crucial for latency-critical SCADA or real-time systems.

Conclusion: Bridging the Worlds

Cilium Cluster Mesh offers the perfect balance for critical infrastructures: The control plane remains separate (maximum resilience against cluster failures), but the data plane is securely networked (maximum flexibility). It makes the network “invisible” to the application and “watertight” for the auditor through comprehensive visualization (via Hubble).

FAQ

Does the mesh create a dependency between clusters? No. The mesh is designed so that each cluster remains autonomous. If the connection between sites fails, each cluster continues to operate locally without interruption. Only cross-site communication is interrupted, which does not affect local availability.

Do the IP ranges of the pods in the clusters need to be different? Yes, a non-overlapping IP concept (Pod-CIDR) is required for a functional Cluster Mesh. We ensure this through careful network planning in advance.

Is Cilium Cluster Mesh harder to debug than traditional networking? On the contrary. With the “Hubble” tool, Cilium provides a graphical overview of all network flows. You can immediately see which service has rejected a connection or if a network policy is blocking access.

What is the additional latency caused by the mesh? The mesh itself adds almost no latency. The delay primarily results from the physical distance of the data centers (signal travel time in fiber optics). eBPF ensures that packet processing on the servers remains highly efficient.

How does ayedo support the introduction of Cilium? We handle the migration of your existing network to Cilium, configure the Cluster Mesh, and implement your security policies as “Network Policies as Code.” We ensure that your site networking is KRITIS-proof and low-maintenance.