Optimizing Pod Assignment: QueueingHint in Kubernetes v1.32

The Kubernetes Scheduler is the core component that determines which nodes will run new pods. It processes these new pods one at a time. Therefore, the scheduler’s efficiency becomes increasingly important as your clusters grow.

Over the years, the Kubernetes SIG Scheduling has optimized the scheduler’s throughput with various improvements. In this article, we introduce a significant enhancement to the scheduler in Kubernetes v1.32: a scheduling context element called QueueingHint. Here, you’ll learn how QueueingHint improves the efficiency of pod assignment.

Scheduling Queue

The scheduler stores all unassigned pods in an internal element known as the scheduling queue.

The scheduling queue consists of the following data structures:

• ActiveQ: Contains newly created pods or pods ready to be reprocessed for assignment.
• BackoffQ: Contains pods ready to be reassigned but must wait for the end of a backoff period. The backoff period depends on the number of unsuccessful assignment attempts the scheduler has made for that pod.
• Unschedulable Pod Pool: Contains pods that the scheduler cannot assign for one of the following reasons:
  • The scheduler has previously attempted to assign the pods but was unable to do so. Since that attempt, the cluster has not changed in a way that would make these pods assignable.
  • The pods are prevented from participating in assignment cycles by PreEnqueue plugins, for example, they have a scheduling gate and are blocked by the scheduling gate plugin.

Scheduling Framework and Plugins

The Kubernetes scheduler is implemented according to the Kubernetes Scheduling Framework. And all assignment functions are implemented as plugins (e.g., Pod Affinity is implemented in the InterPodAffinity plugin).

The scheduler processes pending pods in phases called cycles:

1. Scheduling Cycle: The scheduler takes pending pods from the ActiveQ component of the scheduling queue one at a time. For each pod, the scheduler executes the filter and scoring logic of each scheduling plugin. The scheduler then decides which node the pod should be placed on or if the pod cannot be assigned at that time.

   If the scheduler decides that a pod cannot be assigned, that pod enters the Unschedulable Pod Pool of the scheduling queue. However, if the scheduler decides to place the pod on a node, the pod enters the binding cycle.

2. Binding Cycle: The scheduler communicates the node placement decision to the Kubernetes API server. This process binds the pod to the selected node.

Apart from a few exceptions, most unassigned pods enter the Unschedulable Pod Pool after each scheduling cycle. The Unschedulable Pod Pool component is crucial because the scheduling cycle processes pods individually. If the scheduler constantly had to attempt to place unschedulable pods instead of passing them to the Unschedulable Pod Pool, many scheduling cycles would be wasted on these pods.

With the introduction of QueueingHint in Kubernetes v1.32, developers and DevOps teams can significantly enhance the efficiency of their pod assignment processes. ayedo supports you in making the most of Kubernetes and optimizing your cloud infrastructure.

Source: Kubernetes Blog