Kubernetes Garbage Collection: Cleaning Up Terminating Pods
• 1 Minute Lesezeit

Kubernetes Garbage Collection: Cleaning Up Terminating Pods

Kubernetes life hack - Say goodbye to ‘Pod stuck in terminating state’. Whether volume mounts fail or new replicas hang, a simple Kubernetes CronJob is enough to clean up endlessly terminating pods.
kubernetes

Operating your apps in Kubernetes is a breeze with ayedo. However, sometimes the built-in tools aren’t enough to ensure the smooth operation of your applications. In this article, we introduce a handy little life hack related to terminating pods. Don’t have a Kubernetes cluster yet?

With ayedo Fleet, you can test Kubernetes for 30 days free of operational headaches. Give it a try.

Kubernetes Garbage Collection

The CronJob:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: garbage-collector-cronjob
  namespace: kube-system
spec:
  schedule: '*/1 * * * *'
  concurrencyPolicy: Forbid
  suspend: false
  jobTemplate:
    metadata:
      creationTimestamp: null
    spec:
      parallelism: 1
      completions: 1
      backoffLimit: 0
      template:
        metadata:
          creationTimestamp: null
        spec:
          volumes:
            - name: scripts
              configMap:
                name: garbage-collector-cm
                defaultMode: 320
          containers:
            - name: garbage-collector
              image: alpine/k8s:1.25.12
              command:
                - /scripts/remove-evicted-pods-all-ns.sh
              resources: {}
              volumeMounts:
                - name: scripts
                  mountPath: /scripts
              terminationMessagePath: /dev/termination-log
              terminationMessagePolicy: File
              imagePullPolicy: IfNotPresent
          restartPolicy: Never
          terminationGracePeriodSeconds: 30
          dnsPolicy: ClusterFirst
          securityContext: {}
          schedulerName: default-scheduler

And the corresponding ConfigMap:

#!/bin/bash
namespace=""
node=$(kubectl get node | grep "worker" | grep "NotReady" | awk '{print $1}')
for pod in $(kubectl get pods -A -o=JSON --field-selector spec.nodeName=${node} | jq -r '.items[] |
 select(.metadata.namespace | startswith("${namespace}")) | .metadata.name') ; do
  namespace=$(kubectl get pods -A -o=JSON --field-selector metadata.name=${pod}| jq -r ' .items[] .metadata.namespace')
  echo "Killing pod ${namespace}/${pod}"
  kubectl delete pod -n ${namespace} --force ${pod}
done
Vorheriger Post Alle Posts Nächster Post

Ähnliche Artikel