Kubernetes in the Air Gap: How to Start a Cluster Without Internet

Ever wonder how software gets deployed onto a system that is deliberately disconnected from the Internet and other networks? These systems are often isolated from the Internet due to their sensitive nature. This includes utilities (electricity/water), banks, healthcare, weapon systems, and other government applications. Sometimes it’s even a moat, like when running Kubernetes on an underwater vehicle. Yet, these environments still need to be operated with software. This concept of deployment in an isolated state describes what it means to deploy to the other side of an air gap.

Despite this separation, software must run in these environments. Traditionally, software artifacts are physically transported across the air gap on hard drives, USB sticks, CDs, or even floppy disks (for older systems). Kubernetes is particularly well-suited for operating software behind an air gap, primarily due to its declarative nature.

In this blog post, I will walk through the process of bootstrapping a Kubernetes cluster in a tightly isolated lab environment using Fedora Linux and kubeadm.

Setting Up the Air Gap VM

A truly isolated network can take some effort, so for this post, I will use a sample VM on a laptop and make some network changes. Below is the topology:

Local Topology

This VM will have its network connection disabled, but in such a way that the VM’s virtual network card is not shut down. Instead, the network is disabled by inserting a default route to a dummy interface, making anything internet-based unreachable. However, the VM still has a connected route to the bridge interface on the host, meaning network connectivity to the host still works. This configuration allows data to be transferred from the host/laptop environment to the VM via scp, even when the default route on the VM blocks all traffic not destined for the local bridge subnet. This type of transfer is analogous to carrying data across the air gap and is used in this post.

Further details on the lab setup:

VM Operating System: Fedora 37
Kubernetes Version: v1.27.3
CNI Plugins Version: v1.3.0
CNI Provider and Version: Flannel v0.22.0

Although this simple VM lab is a simplified example, the diagram below shows what a real air-gapped environment might look like:

Please note that there is still an intended isolation between the environment and the Internet. There are also some things not shown to keep the diagram simple, such as malware scans on the secure side of the air gap.

Back to the single VM lab environment.

Source: Kubernetes Blog