Kubernetes
Intermediate 14 min read Kubernetes

Nodes, Node Pools & Autoscaling

Two scaling loops: pods and nodes

Workloads scale via the pod autoscaler (CPU/memory/custom metrics); clusters scale capacity via node autoscalers adding/removing pooled instances. Together they react to load predictably — if configured and budgeted.

HPAVPACluster Autoscalernode pooltaintstolerationsrequestsscale
Prerequisites: Kubernetes Architecture
Before we start

What you will learn

  • Configure Horizontal Pod Autoscaler (HPA) on key metrics
  • Explain Cluster Autoscaler node growth/shrink
  • Use requests/limits + node pools to stage scaling
In 60 seconds

HPA watches a metric (usually avg CPU utilization vs requests) and grows/shrinks Deployment replicas; VPA recommends/adjusts requests; Cluster Autoscaler adds/removes nodes when pending pods or idle capacity exists. Nodepools = label sets of machine types you scale independently.

Introduction

What is this?

HPA: metrics (avg CPU% of requests, custom/external metrics), target, scale-up stabilization (defaults), minReplicas/maxReplicas, behavior field (stabilizationWindow, policies). VPA: Recommender/Updater (updates requests; reboots the pod) vs limited off support; recommended alongside HPA with off/mode alignment. Cluster Autoscaler: scans unschedulable pods → scales node pools; scale-down conditions (underutilized ≥ N min, allows-during-core hours). Node pools: nodeSelector/nodeAffinity + taints (NoSchedule) → workloads tolerate; budgets: min/max nodes, spot vs on-demand, GPU. Pod-level: requests define autoscaling headroom; limits cap and evict. Also regional capacity: priority/spot crashes change CA behavior.

Motivation

Why do you need it?

  • Single scaling doesn't exist — pods can't run without node capacity; nodes can't fill requests.
  • Over-requesting memory starves autoscaling signals; under-requesting causes reliability surprises.
  • Spot node pools cut cost by 60-80% when crash-tolerance is designed in.
  • Finely-tuned stabilization avoids thrash between pod and node autoscaling.
Think about it

Mental model

Imagine

A restaurant hiring chefs (pods) and renting more kitchens (nodes) per demand

Think of

HPA + cluster autoscaler

At lunch rush you add chefs (HPA) until the kitchen is full, then rent another kitchen (Cluster Autoscaler adds a node). Off-hours, kitchens (nodes) close and chefs (pods) go home.

Mechanics

How it works

  • Metrics server samples cpu/memory; HPA computes desired replicas = ceil(current × target/current).
  • If unschedulable pods remain (no capacity on any node), Cluster Autoscaler requests a node from the pool.
  • New node joins via cloud provider integration → kubelet registers → scheduler fills it.
  • Scale-down: over long idle windows CA drains nodes to consolidate, respecting PodDisruptionBudgets.
  • Spot instances can be reclaimed; design stateless + interruption tolerant to ride them.

Two-stage scaling.

Load spike
HPA
Unschedulable
Cluster autoscaler
Idle shrink
Hands-on

Examples

HPA with stabilization

HPA with stabilization
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: { name: api, namespace: prod }
spec:
  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: api }
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource: { name: cpu, target: { type: Utilization, averageUtilization: 60 } }
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300   # don't sprint to 0

Scale out on sustained CPU, hold scale-down for five minutes to absorb traffic valleys.

Real world

In production

  • HPA on a CPU+memory workload with maxReplicas = reserved quota.
  • Spot pool tagged for batch + HPA-eligible stateless services; on-demand pool for stateful/core.
  • Cluster autoscaler with min/max per pool and PDBs so drains don't kill in-flight requests.
Guidelines

Best practices

  • Size requests from real profiling, not guesses; HPA signal quality depends on requests.
  • Set generous scale-up and conservative scale-down stabilization to avoid thrash.
  • Put PDBs on anything with a rollout; CA honors them.
  • Separate node pools by workload class (spot/batch vs on-demand/core) and taint accordingly.
  • Alarm on pending pod count and CA spend changes — silent scale-down eats SLA.
Watch out

Common mistakes

  • Setting HPA on CPU when the app is memory-bound → never scales correctly.
  • No maxReplicas on an unbounded spike loop into cloud-bill inertia.
  • Cluster Autoscaler scaling in every few minutes because stabilization is off.
  • Spot with zero crash tolerance → sudden janitor-style service gaps.
When things break

Debugging tips

  • Pending pods in cluster → check unschedulable reasons: capacity, taint, affinity, quota.
  • HPA does nothing while CPU is 90% → requests are wrong or metric name/unit mismatched.
  • Scale-down pauses → CA awaits PDB rest; check 'force' count and node-grace.
  • Node pool filled on onboarding but never drained → affinity/PAB blockers.
Interviews

Interview checkpoint

Practice

Hands-on lab

Scale Pods with the HPA on Real CPU
35 min · intermediate
In progress
Objectives
  • Install metrics-server and HPA
  • Create load and watch replicas grow
  • Remove load and watch them shrink (slowly)

Scale Pods with the HPA on Real CPU

35 min · intermediate

0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

HPA average CPU% is computed against:

2

Which prevents CA from removing a node mid-request-drain?

Go deeper

Official documentation

HPA
docs
Cluster Autoscaler
docs
VPA
docs