K3s on Flatcar via Ansible & vSphere: Automated K8s for Regulated Environments

In industries like manufacturing, finance, or critical infrastructures, automation is not a “nice-to-have” but a mandatory necessity. Kubernetes is well-established – yet the provisioning of control planes in regulated on-premises environments remains complex.

Virtual machines must be auditable, reproducible, and secure. Standard templates with Ubuntu or CentOS are often challenging to keep up-to-date. At the same time, compliance requires full control over supply chain, secrets, and network access.

An approach particularly suitable for edge, test, dev, and even prod clusters in sensitive environments: Flatcar Linux + k3s, fully automated provisioning via Ansible on vSphere.

Why Flatcar + k3s?

• Flatcar Linux is a minimal, container-optimized Linux fork (successor of CoreOS). No bloat, automatic updates possible, and fully configurable via Ignition.
• k3s is a lightweight, CNCF-certified Kubernetes distribution from Rancher. Optimized for edge and CI/CD, small footprint, yet fully compatible.
• Ansible orchestrates the entire lifecycle – from VM creation in vSphere to handing over the Kubeconfig.

The result: quickly deployed, standardized control planes, without click-ops in vCenter, reproducible from code, and adaptable for air-gapped environments.

Architecture Overview

1. vSphere provides the foundation: datacenter, cluster, network, datastore.
2. Ansible with the community.vmware collection controls VM creation & configuration.
3. Flatcar OVA serves as a template – Ignition config is injected via guestinfo.
4. Ignition sets up SSH keys, systemd units, and k3s services.
5. k3s starts as a control plane, either as a single server or as an HA cluster with embedded etcd.

Why not Ubuntu + Cloud-Init?

In many industries, the drawbacks of Cloud-Init VMs are well-known:

• Templates must be manually patched regularly.
• Differences in user-data processing per version/cloud driver.
• No deterministic, reproducible build.

Flatcar + Ignition offers:

• Immutable images, configuration via clearly defined JSON descriptors.
• Easy integration into GitOps: Versioned Ignition JSON = reproducible cluster.
• No dependency on distributed Cloud-Init implementations.

Ignition: Infrastructure as Code

Flatcar reads an Ignition JSON file at boot, provided via VMware guestinfo. This describes users, files, units, network – and thus the k3s bootstrapping process.

Example: k3s Server (Single)

{
  "ignition": { "version": "3.4.0" },
  "passwd": {
    "users": [
      {
        "name": "core",
        "sshAuthorizedKeys": ["ssh-ed25519 AAAAC..."]
      }
    ]
  },
  "storage": {
    "files": [
      {
        "path": "/etc/systemd/system/k3s.service",
        "mode": 420,
        "contents": {
          "source": "data:text/plain;charset=utf-8,[Unit]\nDescription=Lightweight Kubernetes\nAfter=network-online.target\n..."
        }
      }
    ]
  },
  "systemd": {
    "units": [
      {
        "name": "k3s.service",
        "enabled": true
      }
    ]
  }
}

Advantage: complete config versioned in the Git repo. Every re-deploy is identical.

Ansible: Full Automation on vSphere

With the Ansible collection community.vmware, VMs can be deployed from content library OVAs, Ignition injected, and IP addresses queried.

Example: Single k3s Server

- name: Provision Flatcar VM and run k3s (single server)
  hosts: localhost
  collections:
    - community.vmware
  tasks:
    - name: Deploy VM
      vmware_content_deploy_ovf_template:
        hostname: vcsa.lab.local
        username: svc_ansible@vsphere.local
        password: "{{ vcenter_password }}"
        datacenter: "DC1"
        cluster: "Compute"
        datastore: "vsanDatastore"
        name: "cp1"
        content_library: "BaseImages"
        template: "flatcar-stable-vmware.ova"
        folder: "/DC1/vm/k3s"
        networks:
          - name: "VM Network"
        power_on: true
        advanced_settings:
          - key: "guestinfo.ignition.config.data"
            value: "{{ lookup('file', 'ignition-k3s.json') | b64encode }}"
          - key: "guestinfo.ignition.config.data.encoding"
            value: "base64"

HA Control Planes: 3× Flatcar + k3s

For production scenarios, a 3-node cluster with embedded etcd is recommended:

• Node A: --cluster-init
• Node B/C: --server https://<VIP>:6443 --token <token>

Load balancer (VIP) can be provided via kube-vip or keepalived.

This creates highly available control planes that can be provisioned in minutes – ideal for tenant clusters or staging systems.

Operation in Regulated Industries

In industries with high compliance requirements, details matter:

• Air-Gapped: No download from get.k3s.io, but own mirror. Ansible replaces the URL in the Ignition template.
• PKI & Secrets: Distribute kubeconfigs & certificates via Vault or HSM-secured PKI.
• Audits: Every VM definition is documented in Git. Change management: PR → audit trail.
• Update Strategy: Flatcar channel (stable) or controlled rolling upgrades via Ansible playbook.
• Storage & Backup: Automate etcd snapshot backups, provide recovery playbooks.

Example: CI/CD & E2E Tests

A particularly strong use case: end-to-end tests per pull request.

• PR is created → Ansible starts Flatcar VM + k3s control plane.
• App is deployed into the cluster.
• Cypress/K6 tests run in an isolated environment.
• Logs & reports are secured.
• VM is deleted again.

This is reproducible, parallelizable, and without a shared cluster – perfect for fail fast, fix fast.

Alternatives Compared

• Kamaji (kamaji.clastix.io): Hosted control planes in Kubernetes itself, ideal for SaaS platforms.
• Gardener (gardener.cloud): Full-fledged cluster-as-a-service platform, suitable for hyperscaler-like operators.
• k3k (ayedo/k3k): Control planes in Kubernetes, extremely fast for multi-tenant & CI.

In comparison, Ansible+Flatcar+vSphere offers advantages especially in traditional data centers and regulated industries, as it integrates seamlessly into existing VMware ecosystems – without an additional platform layer.

Conclusion

With Ansible, Flatcar, and k3s on vSphere, Kubernetes control planes can be provided in regulated environments standardized, securely, and reproducibly.

The result:

• Fast rollout of new clusters – whether for tests, staging, or edge.
• Compliance-compliant through GitOps, immutable OS, auditability.
• Scalable from single server to HA control plane.

Anyone operating Kubernetes in industry or regulated sectors today needs exactly this: automation with clear governance – not click-ops.

Further Reading

• k3k: agent-less k3s in Kubernetes
• Developer Platforms by ayedo
• Governance & Security for AI Teams in Kubernetes