2025-01-14

Using a Kubernetes Admission Webhook to Block Workloads When Capacity Is Not Available

A generalized design pattern for validating workload capacity before admitting tenant workloads into a Kubernetes cluster.

KubernetesGoAdmission Webhooks
Using a Kubernetes Admission Webhook to Block Workloads When Capacity Is Not Available

Kubernetes will happily accept a workload that cannot run. The scheduler may leave it pending forever, but by then the user already believes the platform accepted their request.

A validating admission webhook can move that failure earlier.

Instead of accepting a workload and discovering later that it cannot be scheduled, the platform can reject the request with a clear reason.

The basic pattern

The webhook receives an admission review for a workload type, evaluates the requested resources, compares them to available capacity and policy, then returns allow or deny.

At a high level:

User applies workload
        ↓
Kubernetes API server calls validating webhook
        ↓
Webhook checks requested CPU, memory, GPU, labels, and policy
        ↓
Webhook returns allowed=true or allowed=false with a reason

What to validate

The exact logic depends on the platform, but useful checks include:

  • requested CPU
  • requested memory
  • requested GPUs
  • node selectors
  • tolerations
  • required labels
  • namespace or tenant policy
  • current allocatable capacity
  • reserved capacity
  • platform safety buffer

Why this is useful

A good denial message is better than a silent pending pod.

Example:

Denied: requested 4 GPUs, but only 2 GPUs are currently available for this workload class.

That gives the user an answer immediately and avoids creating broken environments that someone has to clean up later.

Make the webhook boring

Admission webhooks sit directly in the API request path. That means they should be:

  • fast
  • highly available
  • easy to observe
  • conservative
  • explicit about failure behavior

For most capacity guardrails, I prefer strict failure behavior only after the webhook is stable and well monitored.

The bigger lesson

Admission control is not just a security feature. It is also a platform usability feature.

When done well, it turns confusing runtime failures into immediate, understandable feedback.